Wednesday, September 20, 2017

#Data17 Survival Guide



Just a few weeks to go before data nerds converge in Las Vegas for the 10th Tableau conference. This will be my 7th and I still get excited about it.


  1. Have fun - There will be parties, mingle time, relax time, keynotes (even those are fun). Welcome Reception on Monday and Empire of the Sun and Robert DeLong will be playing at Data Night Out on Wednesday. 
  2. Meet people - in the elevator, in sessions, at the parties, in the hall. Go to meetups and meet people from the same industry or in similar roles. 
  3. Comfy shoes - you are going to walk. 
  4. Travel light - I hated carrying a bag with my laptop and all the trimmings. It kept getting in the way during sessions and I had to keep track of it. I tried slimming down to an iPad, but now I just go with sunglasses, a phone, and a power pack. Everything fits in my pocket so I can. I feel bad for those folks with computer bags at Data Night Out. 
  5. Be flexible - You might go to a session, but it isn't your cup of tea. Don't be afraid to bail and check out our second choice, or go do something else. 
  6. Tweet - The main hashtag is #data17, but there are others, like #rundata17, #fitdata17, #DataNightOut, #data17wish and more. 
  7. Eat and drink - We will be in the desert, so keep hydrated during the day so you can party in at night. Pro tips:
    1.  here is the Bar Chart often starts serving beer and wine around 3
    2. Often people line up funny at the food tables. It isn't a buffet and the same food ready to grab is all over the table, but people form a line at one end and just take food from one side. Mob the table people!
    3. Drinks lines can be crazy, so go to the back of the party where they are shorter. 
  8. Pick the right sessions - I usually star everything I am interested in and then remove anything I think will be as good or better in the video replay (you do know they record the sessions?). Hopefully, I have two or three for each timeslot to give me a backup if something is full or boring. Some people like the hands-on sessions, but I am not one of them. They are longer and in my experience, I don't get as much out of them.
  9. Dress how you want - Anything from a suit to jeans and t-shirt is acceptable. Average Highs are between 82-84 and lows around 60. Record temps are high 90's and mid-30's. Check the weather before you go to pack right. 


Tuesday, September 19, 2017

My Ski Data

Last year I inadvertently tracked my skiing for a week at ABasin using the Moves App. It wasn't super accurate and lacked elevation (a big deal when skiing). This past season I used an app called Ski Tracks that is designed for skiing. It is also smart enough to split runs and chairlift rides. It started snowing out west this weekend so I figured I should get this finished up.

Here are the results of my 21 ski days for the 2016-2017 season.



Details:
The app I used will export as KML or GPX, but I found the .SKIZ files it uses are actually .ZIP files. They contain an XML with day stats and CSV files with run segments, nodes, battery life, and any photos I tagged along the way.

I created a pentaho job that would expand the SKIZ files, and then process them into 3 files.

  1. Day Level Stats - one record per file/day that has max speed, total decent, number of runs, min and max altitude, etc. 
  2. Ski Paths - a combination of the runs and nodes data. Along the way I also add some calculations for delta altitude and distance so I don't have to do table calcs in Tableau. 
  3. Battery life - it tracks battery life during use and will stop tracking if the battery gets low, but I am never reached that point or have done anything with this data yes. 
Once the data is processed I take it away with Tableau! For my detailed speeds, I ended up using a moving average. I have found that with GPS data at small distances can be...less than accurate. Some line segments had me over 150mph. I am good, but not that good. A moving average helps smooth the speed line out. 

Thursday, August 17, 2017

Ticks and Trips: Tick Danger along the MI North Country Trail

We were planning the most recent UMich Tableau Users group and I heard that I was supposed to do a Ticks and Trips. Apparently, they actually said "Tips and Tricks" but my mind was already working. In no time I was combining Lyme Disease data from the CDC and journal articles with demographics data. In order to span much of the state I used the North Country Trail as my "trip" and I was off. I built this on the fly, incorporating many tips and tricks. One person in the audience raised her hand and told us that she got Lyme Disease hiking in the Western UP, exactly where my viz showed the highest probability.

Monday, January 23, 2017

Great Lakes Ice Cover

I had a discussion the other day about ice cover on the Great Lakes, which led me to searching out some data. Turns out the GLERL here in Ann Arbor has it, a lot of it. After I got it all downloaded and converted the shapefiles to CSV I loaded it up. Wow...there is a lot of it. about 150,000 rows per day, after removing land values, and I had from the 02-03 to 15-16 winters. Turned out to be about 180M rows, way more than the Tableau Public limit of 15M. I picked about 60 random days (8.2M rows) and included just them.
I tend to over complicate things and have to work hard to simplify, simplify, simplify!



Thursday, June 9, 2016

Milwaukee M18 Battery Value

Following this post over at ToolGuyd I did some copy/paste and came up with this analysis. I came to the same conclusion that the 5Ah batteries are the best value.



Thursday, April 21, 2016

Where are you going to stay at #data16?

The Tableau Conference keeps getting bigger and bigger. Gone are the days when it fit in a single hotel. Now you have choices! To help decide which hotel to stay at (JW Mariott) I put together a viz containing location and hotels.com and tripadvisor reviews. You can decide on the right combination of distance, cost, and ratings. Some of the hotels were booked up when I tried.



Monday, April 11, 2016

Spring Break Skiing


Over spring break my family went skiing in Colorado. When we got back I was looking at Moves and realized that it had been tracking my skiing. Fun with data below...





Recently I got a new iPhone. I have used some fitness tracking apps in the past, but my new phone (iPhone 6s) has a built-in pedometer and supports the app Moves. Since 2012 I have used OpenPaths but that is sporadic and only useful for general location.

Moves only recognizes walking, cycling and "transport" automatically. I had to reclassify some of the movements as 'downhill-skiing'. Since they were all in a tight geographic cluster it was pretty quick and easy using the app on the phone. The bummer is that it didn't differentiate riding the chairlift from skiing. Sometimes a movement was just a run or a chair, but most were multiple runs and chairs rides.

 Ski data is all about elevation, but there was no relevant data captured. I dumped the timestamp, lat, and long into a webpage that can calculate elevation and added that as a second dataset. Next time I might use one of the ski specific tracking apps if better ski tracking isn't added to Moves.

Next step was to get it into Tableau, but the folks at Interworks already created a WebDataConnector for it, so that was a breeze.