This is still an ongoing project, but this particular setup was mostly an exploration tool to get oriented with some of the data. About a year ago, I started collecting GPS tagged tweets. Collection is continuous and fairly robust to periodic network outages or receiving bad data. It is about 1 GiB of data/hour...and we just passed the year mark! Every 8 hours, we save the currently accumulating data to file, which eventually makes its way into MongoDB. The goal of the project is to study the ways in which Twitter has transformed our familiar geographic landscape. It's quite a fun problem theoretically, and the data is quite messy.
In this display, we visualize various scalar valued datasets for every county in the contiguous states. We also did this with congressional distributions, states, and an arbitrary grid, but eventually settled on counties as our target "object". This was also my first big push into MongoDB.
Technologies: python, HTML, JavaScript, Google Maps API, MongoDB, topoJSON, Twitter, tweepy
Back in 2013, I started playing a mobile phone game called Parallel Mafia. A major component of the game is going to "war" against other mafias. The game players were from all over the world, and one tested, successful war tactic was to attack when most of the other mafias' players were sleeping.
Bored one weekend (maybe two), I started scraping online status information from the community website. The was no API, and I had to download a PNG icon for each player and look for a green dot. Oh my. The data was obtained every 15 minutes of every hour, and is still being collected to this day, even though I no longer play.
The goal was to gather information about player activity and visualize it. It tracks the average number of online players---overall, as a function of day per week, by mafia coalitions, etc. Additionally, you can see per player heatmaps indicating the probability that they would be online; it's definitely heavy-tailed with some users just a bit on the insane side. I didn't put the time in to do any advanced recommendations for attack times, but our mafia was able to make use of this tool to plan and coordinate ideal attack times with great success!
Technologies: python, HTML, JavaScript, CSS, jQuery, flot.js, tablesorter, unicode, sqlite, cron, git
In this project, user's browsers are turned into "digital cicadas". Well, that was the hook line, at least. Users activate their microphones which listen and then playback the sounds they hear. Now imagine 30+ laptops all doing this at the same time. The sound each cicada hears is a influenced strongly by those around it. Under certain conditions, the cicadas will reach consensus on the frequency of the sound that is generated.
At the backend, there is a very hackish client-server implementation. We needed to be able to control every user's display (see here). So the client-side periodically queries some WSGI python server which prepares a JSON-encoded representation of what the admin control page had specified. We also plot the frequencies of each active cicada as a function of time, and visualize their geometry as a spatial heatmap of frequencies.
Technologies: python, HTML, JavaScript, CSS, jQuery, p5.js, d3.js, flot.js, heatmap.js, sqlite, git
This was a fun project designed for a high schoolers in a science outreach program. The idea was demonstrate to students how worms with relatively limited resources might go about finding food. In this demo, there are 4 worms types whose names are characters from Saved By The Bell. Unsurprisingly, none of the high schools caught this reference. Each worm type has a different "vision" ability, and the game is to find the place with the most food in a given number of steps. It starts off easy with an energy landscape having a single maximum, but if you click on the "Hard" mode at the top right, you go to a much more difficult landscape. Try it out. Rank the worms in terms of their ability to find food.
A challenge for students is to really "think" like the nematode and not use their super-human ability to find food easier. Data on the maximum energy level achieved is stored after each game. The current tallies, as a function of worm type, are plotted in a histogram via the link at the top right of the page (link). This histogram is generated every minute using the most recent data. If you want to see the solution for the "Hard" puzzle, click here and then select "Show Map". You'll see that the landscape had multiple local maxima. The led to fruitful discussions on exploration vs. exploitation.
Technologies: python, HTML, JavaScript, CSS, jQuery, d3.js, MySQL, PHP, matplotlib, cron, git