Learning About Messy NJ Data through Hacking at Hack Jersey 2.0

Part of my job as a contributor to NJTechWeekly.com is to cover hackathons around the state and to highlight the most interesting projects that emerge from these overnight coding-fests. I will admit that I have harbored a bit of envy for those who participate. The kinds of projects they present are challenges I am drawn to try to solve, but I never felt of age, qualified, or focused enough to participate.

But there is one hackathon in particular that I could not resist participating in based on the audience it was looking to attract. They wanted people of any skill or age level, passionate about journalism, and have a love for New Jersey.

This hackathon, Hack Jersey 2.0, held at Montclair State University on March 25th and 26th, is the brain child of Baristanet founder and associate director of the NJ News Commons at Montclair State, Debra Galant, and data journalist at The Marshal Project Tom Meagher.

They held their first event back in 2013 that brought together some of the best coders and journalists in the New Jersey area. The event marked the first news hackathon in the state. Given the dilapidated state of local news in New Jersey, the duo wanted to show how data journalism can be applied to New Jersey.

The year 2015 saw the second iteration of the event. Having covered many hackathons in the past, I did not consider myself a total amateur — I know how hackathons run and what is expected of each team — but still amateur enough that I don’t know how a programmer works. If my programming skills could be judged by the wear and tear of my O’Reilly “Learning Python” book, I could sell it mint on the Amazon Marketplace.

My team consisted of a journalism professor, a professional programmer, and three MSU students of varying programming experience. None of them had experience in a hackathon, but were enthusiastic enough that I knew I was paired with a good group of people.

My skills lie in my past as a local journalist in Essex County and as a policy researcher in the New Jersey statehouse where I developed my knowledge of how local and state government works. What I could not bring in programming abilities I brought in my specialized understanding of New Jersey issues.

Within the first hour after we started at 10:30 am we settled on a topic: mapping all the structurally deficient bridges around New Jersey. The state’s bridges ranks 6th worst state in the U.S. with 35.45 percent of all bridges considered “deficient”

The dwindling Transportation Trust Fund that pays for road and bridge project and the controversy on how finance this trust fund adds to the newsworthiness.  A map where drivers can see the condition of the bridges they frequently travel on can inform them of the need for the repairs.

There was one problem: NJ Spotlight’s data wizard, Colleen O’Dea, had already done such a project for that news organization’s website.  At this point it was already 2:30 pm, and we had to have our project idea recorded by 3:30 pm. We scrambled to find a new project to create. It had to be something where the data was easily accessible and could easily be parsed with little cleanup. After some intense brainstorming we decided to use election donation data in New Jersey legislative races to see if there were any longitudinal trends in individual donations versus organization and political action (PAC) donations. And no, Colleen (as far as we knew) did not do it. We also came up with our team name during this period: “Colleen Did It.”

Once we settled on the topic and gathered the data we took to our roles. The programmers worked their magic loading the data into a database while I filled my role as a journalist to help contextualize the data.  We grew to learn a lot from each other. Even though my skills in programming were limited, I helped them understand New Jersey’s legislative districts and how they are represented in the statehouse.

After a while I left them to their typing and observed how they put the parts together. That day’s session stopped at 7:30pm for food and drinks at a nearby restaurant. The teams would pick up the next day to continue their work, or continue through the night on their own time if they so choose.

Unfortunately, my utility to the group was cut short when I developed a nasty bout of laryngitis at the end of the night, robbing me of my voice for the rest of the week. I ended up helping my team put together their presentation from my bed via Slack and Google Drive the next day.

When the judges decided on the winners, my team did not win a prize. While it is a bummer we did not end up winning, the experience working with such interesting people was enough of a reward in itself. I feel I more clearly understand the skill that goes into coding projects and the obstacles that can emerge.

The biggest takeaway from the event according to the presenters was how messy New Jersey’s data is, saying they would have added more details to their projects if they did not have to waste time cleaning up the data. State data is notoriously disorganized with much of it meeting the bare minimum requirements put forth in New Jersey’s public transparency laws. Perhaps if more forward-thinking lawmakers attended this event and others like it, they would see the necessity toward updating the state’s plodding transparency initiatives.

Would I participate in a hackathon again? Definitely. Though I would not participate in just any hackathon I come across. It is important to choose a hackathon where the theme matches your interest.

Even if you lack technical skills, odds are you will be more of an asset to your team then you think. You can bring a specialized knowledge to the team that could give you the leg up against other teams.

For example, if you work in finance and you come across a hackathon focused on financial tech (aka Fin-Tech), a lack of programming or technical prowess will not make you any less of an asset; it is likely your technically literate teammates are equally unfamiliar with finance.

The real reward of such events is not the prize, but the ability to network with innovate like-minded people and an understanding of how many of the digital tools you use every day are created.

[Steven Galante has been a contributor to NJTechWeekly.com since 2013.]