I recorded every step I took for over three years

Last year I published an analysis of the steps I took over the past 2 years, this year I decided to update with the newest data from 2017.  I’ve seen my daily steps decrease by 17% to 8,024.  My daily goal was to reach 10k steps, but I only managed to achieve that 83 days (23% of the time).

Temperature

Last year I pointed out how the temperature has an effect on the number of steps I take.

123% more likely to take +15k steps when it’s warmer.

143% more likely to take less than 2,000 steps when it’s cold.

Location

In June I visited London and Europe to see the French Open.  I tried to pack as much into the trip as possible, which meant I spent most of my time walking.

Back in America, I noticed that living near a subway station in Brooklyn has drastically decreased the steps I take compared to my old apartment in Manhattan’s East Village which was 1/2 mile from the closest subway.  I now average 7,978 daily steps when I start my day in Brooklyn.

YoY, I’ve seen my daily steps decrease 17% which I attribute to the decrease in +10k step days.  In Brooklyn 24% of my days were 10k steps or more, in Manhattan, that amount was double, with half of my days hitting 10k.

Week Days

What’s my LTV?

Lifetime value is an important marketing concept which describes the amount of money a business can expect earn from a customer.  For this post, I’ve decided to calculate how much I’m worth to a business over the last 12 months using all my spending using Mint.com.

 

Business Analysis

I was really interested in the businesses where I spend the most money.  For this analysis, I have three categories essential expenses costs (rent, public transport, phone, groceries), and non-essential expenses (alcohol, coffee, fast food).  I’ll ignore the essential expenses because nobody really cares that I spend the same amount every month with the MTA or Verizon.   Below is a handpicked list of businesses that I spent lots of money at last year.

BLACK STAR COFFEE, $1,025: The business that surprised me the most was Black Star Coffee.  The average meal for me is a croissant, and a double cortado on weekdays, and a siracha egg sandwich and iced coffee on weekends.  Before looking at the data, I expected my monthly spending to be around $40.   Using Mint, I saw that over the last 12 months, I spent over $1,000 on 136 purchases (average cost of $7.54).  Clearly I was way off, I was blindly handing my credit card over without actually doing the math in my head, of how often I frequented this coffee place.

I like their croissants, and breakfast sandwiches a lot, I highly encourage people to try it out.  But I can’t justify spending at a conservative estimate, $400 a year on coffee from one place.  I could buy a coffee machine for $40, and pay an average of $0.40 for a freshly brewed cup of coffee, saving myself around $3.50 per cup.

TRADER JOES WINE SHOP, $409: Over the last year, I’ve really gotten into Trader Joe’s “two buck chuck”, officially known as Charles Shaw Wine, which is available for $3 for one bottle.  From the data, I make a bi-monthly trip to the Trader Joes wine shop, where I pick up two bottles  the first is usually an $8 Malbec, and the second is either a bottle of two buck chuck, or a second $8 Malbec, all of which leaves me with an average basket of $16.35 (incl. 10% NYC tax).

I expected my spending to be much higher with Trader Joes Wine, I was under the impression I visited more often and spent more in an average visit.  In terms of cost reduction, there’s not much I can except stop drinking.  In my opinion the wine is a great value, it doesn’t taste like it’s two dollars and it’s better than most beer.

Seamless, $714: I was really worried that Seamless would be over $1,000.  I estimated that my average basket was between $18 – $21 (actually $19.85), and that I ordered around 50 times (actually 36 times).  I’m fairly loyal, 2 restaurants accounted for 44% of all orders.  Half of all orders came in the final three months of 2016, while the other half came over the next nine months.  It’s good to know that I learned self control

One interesting to note, in 2016 I spent $979 on 52 orders, I’m not proud of it, but it was so convenient.

Category Analysis:


Fast Food $2,606: This is pretty embarrassing for two reasons. First these costs are mostly avoidable if I brought lunch, or stopped using Seamless.  Bringing lunch twice a week could save $1,000.  Second, I spent less on groceries than fast food.  Every time I buy lunch, I spend an average of $10.47, but cooking at home could cost me around $2 – $3 which is an 80% decrease in costs.

Sports Tickets $1,804: I’m a huge sports fan, over the past 12 months I went to the French Open, US (Tennis) Open, NHL Rangers games, and several soccer games.  Over the next 12 months I expect this to remain the same or slightly increase because I’d like to attend more NHL games, and an NFL game.  It’s not cheap, I spend around $150 dollars on each event (excl. soccer).

Television and Streaming $549: I use Sling TV, HBO Now, and Netflix Streaming for all of my TV and streaming.

Does this satisfy all my viewing needs?  Technically yes, the only thing I’m missing certain sports networks like YES, FS1 and MSG.  The channels that I do watch live are available in Sling and HBO Now: ESPN, Comedy Central, Adult Swim, HBO, and I can watch certain sports using a $15 (one time cost) digital TV antenna.

For comparison, using cable like Spectrum I could pay around $130/month, putting my total spend to $1,560 + $203 from Netflix.

 

Next Steps

Some areas were eyeopening.  I spend way too much on greasy Seamless food and lunch.  If I do spend money, it should be on nicer restaurants that I visit with friends.  Coffee was another unnecessary expense that I regret.  Coffee was usually ordered with a snack or sandwich which often doubled the cost.  For the next 12 months I should focus on brewing my own coffee, eating out less for dinner and bringing my lunch which could easily save me $1,000.  One area that I don’t expect to change, is sports tickets and alcohol.  I really enjoy seeing sports live and I don’t expect to change that anytime soon.  I would like to spend less than $2,000 over the next year, but realistically with dates, happy hours and NYC’s obsession with cocktail bars it will be hard.  I think it makes sense to track these expenses closer in Mint.  I’ve set up budgets which I’m trying to follow a little more closely, and I’ll try and do a quarterly post-mortem of expenses to see if I followed through on the cost reductions.

 

Source: Mint, Excel used for Viz

From Time Magazine to Gizmodo, here are the publishers that were put up for sale in 2018

2018 was a transitional year for publishers who saw reorganizations, sales, and acquisitions.  I took a look at over 75 publishers to see how they performed.

Moving from left to right, Meredith is working on the divestiture of it’s Time Inc. assets.  Time was the most notable, going to Marc Benioff for $190 million in September, quickly followed by Fortune in November for $150 million.

Gothamist had a turbulent 2018.   In November 2017, it was shut down after the editorial team voted to unionize, but it was quickly revived by public radio station WNYC in February 2018.

Picking the best python graphs for beginners – Plotly, Seaborn, Matplotlib, Chartify

Are you new to Python and trying to make a beautiful graph? I’ve reviewed four of the most popular and picked the best option for beginners.  For the cells below, I used Jupyer Notebook with these modules that can be installed via pip (pandas, numpy, plotly, cufflinks, seaborn, chartify).

In a normal day, I’ll open my Jupyter Notebook, import a CSV that I created using SQL/Hive.

remember, this doesn't go in jupyter notebook, it goes in your terminal (the thing with a black screen, sort of looks like that thing from The Matrix)

pip install plotly
pip install cufflinks
pip install chartify
pip install seaborn
import pandas as pd
import numpy as np

%matplotlib inline

import pandas as pd


%cd -q Downloads 
#%cd this changes my directory to the Downloads folder

df1=pd.read_csv('blog_example.csv')
#this uses pandas (pd) to read the csv in the Downloads folder
#this example data mimics Google Ad Manager data, but for this exercise, it's full of random numbers

df2=df1.pivot_table(values='imps',index='day',columns='subset',aggfunc='sum')
#I now have two dataframes: df1, df2. This will be used later, depending on the graph

df2.head()
#.head() will show the first five rows of df2 

Download example data here.

Plot.ly

Link

Learning Curve: Low, my pick for best graphing module for beginners.

What I like: Interactive, easiest library to use for beginners, pretty themes out of the box, other features (export, save as png), easy to understand documentation for new users.

 

What I don’t like: version 2.x is slow.  If you don’t use cufflinks, this becomes one of the most difficult graphing libraries. Requires additional code to run in offline mode.

 

import plotly
import cufflinks as cf

cf.go_offline() 
#cf.go_offline() allows you to use plotly in jupyter

df2.iplot()

Chartify

Link

What I like: Easy to write, built by Spotify Data Science team.

What I don’t like: Requires an additional exe to run (from Google).

 

 

 

import chartify
df=df1.groupby(['day','subset'],as_index=False).sum() 
#chartify can handle a flat table, no need to pivot it

%cd -q
#%cd was needed to change the active directory to 'python', earlier in this lesson I moved it to the Downloads folder. 

ch = chartify.Chart(blank_labels=True, x_axis_type='datetime')

ch.plot.line(
    data_frame=df,
    x_column='day'
    ,y_column='imps'
    ,color_column='subset'
)
ch.show()

Seaborn

Link

pip install seaborn
sns.set()
#sns.set is optional, but I like the formatting
sns.lineplot(x='day',y='imps',hue='subset' ,data=df1,ci=None);

What I like: Pretty visualizations out of the box, great at heatmaps.

What I don’t like: I’ve personally had trouble writing

and remembering the formatting of the plotting functions.

Matplotlib

Link

pip install matplotlib

df1.plot()

Learning Curve:

What I like: Customizable, lots of documentation on StackOverflow

What I don’t like: Difficult to remember all the features.  Learning curve is prohibitive to new users.

I before E except after C is a lie

It’s embarrassing, but I’ve had a lot of trouble spelling the word, “receipt”.  I keep spelling it reciept, which could be avoided if I remembered the simple mnemonic rhyme, “I before E, except after C”.  In this analysis, we see that it’s almost never true.

First, I downloaded a copy of every English word from this github repo, and then running it in Excel, isolated words with the letters “IE” or “EI”.  I found 504 words.

75% of words did not follow the maxim of I before E.

 

Did the first letter in a word impact the I & E order?  It doesn’t seem so.  Only words beginning with B, J, S and V all had an I before an E.

 

Length of the word didn’t have a significant impact on the order of the I and the E.   I & E combinations occurred more in words that had a length between 10 and 12 letters.

 

Williamsburg in seven years

I moved to Williamsburg from the East Village in 2016 because I wanted to pay less rent, shorten my commute and be around a lot of bars and restaurants.  After a year of living Williamsburg, I’ve heard more than my fair share of hipsters gentrifying jokes.  What is interesting about the area is the sense of new-ness in the area.  Looking around, some areas are nothing but apartment building made out of the same metal and glass facade.

Building a time machine with Google

I didn’t live in NYC in 2007, but I am lucky enough to have the next best thing, Google. By using Google Maps’ time machine function.  When in street view, move your cursor to the top left of the screen until you hover over the grey box (in the picture below it says 250 Bedford Ave).  In the bottom part of my box, I can see a clock that labeled, “Street View – August 2007”.  This will open a timeline of every time that Google Street Car has passed by your location.
Note: My goal was to use jQuery to make a before/after effect of the image, but  jQuery and WordPress don’t play well together (it caused my entire site to stop loading), so I published this In a format similar to Business Insider (one big list with next to no insight).

Bedford Avenue

Bedford Avenue is now the heart of Williamsburg, but before that it was full of decaying building.  In the ten years since that photo was taken, an Apple Store, Equinox, Whole Foods and Duane Reade were built in this exact location.


 

McCarren Park Area

This area would be unrecognizable if it weren’t for the houses on the right hand side of the screen.  In eight years, three huge apartment buildings were built in the empty lots and warehouses of East Williamsburg.

 

West Williamsburg

No major architectural changes here, but we can see the cities move to make NYC streets more pedestrian friendly.

Central Williamsburg

This was an amazing picutre.  Before most of the major development we could see the Manhattan skyline between the old buildings covered in graffiti.  In the following eight years, there were new building on every block going all the way to the East River.

Conclusion

I wanted this post to go out to show people how much this area changed in a short timespan. What I would love to look at next is the affect on real estate prices, rent and GDP of the area.