My first hackathon
and the story of how I failed. Twice.
It is my first attempt to write a blog post in English, so don’t be mad at my writing skills and let me speak from my heart!
For me, the ECMWF Hackathon “Innovate with Open Climate Data” was not only the first hackathon but also the first international scientific meeting, and the first visit to the UK. Moreover, in short, it was fantastic!
I am a big fan of the ECMWF’s ERA-Interim reanalysis (in particular) and climate-related studies (in general), so for the last couple of years, it was my dream – to visit ECMWF to see how the most prominent center of hydrometeorological research lives. I did not participate in the first ECMWF hackathon in 2017 – there was too much for me go to England in two weeks before our relocation to Germany. All the time after I was waiting for the next call and I think I had applied for this year’s hackathon on the first possible day.
I do not describe my way to the UK, how I stayed in a row waiting for a border control for 2 hours, how I got lost near Gatwick airport, how I spent a night in a hostel above a bar, how I miss my train to Reading, and how I (in spite of everything) reached ECMWF in time on sunny Saturday :-) Just tell you, that everything I saw was something new for me – from wrong direction of traffic to public transport system to architecture. Now I do not consider the UK as a European country. It is a way different!
Let’s come back to the Hackathon. Who has no idea what is it, it is a meeting where geeks try to make shiny things (like websites, apps, DIY stuff) in accordance to predefined theme staying overnight in some fabulous location with a lot of coffee and snacks. Technically, it is a competition for prizes, but it is more about getting new experience and contacts. In a couple of weeks before the Hackathon, organizers made a call for challenges – new themes you can attack alone or in a team. I had selected one called “Sonifying ERA5 – make the data sing” raised by Florian Pappenberger.
It is entirely a common practice to do some homework before a hackathon, because of limited time participants have for development, and because of a high probability of some random issues might occur, so it is better to have a sketch before. I was so excited about ERA data sonifying that I did an end-to-end prototype before the Hackathon, so a day before I have realized that I need the new challenge to do during the Hackathon! Thank God I met Shaun Harrigan who helped me to harmonize open software and data sources for the idea I have come up.
There was one of the best 24h coding sprints I had – so many interesting people to ask, so many bright people to talk, endless coffee and good weather. Moreover, as a culmination – I have visited the ECMWF Supercomputer room! It is so amazing to know what happens when you call data from ECMWF.
Now, I would like to describe projects I have done for the Hackathon briefly.
In this project, I tried to enrich music generated by MusicVAE model with ERA5 reanalysis data. The core idea is pretty simple – let’s bring a model and try to affect its parameters by ERA5 temperature data (mean and variance). The results were pretty exciting, so for further details follow the main page of the Beats of the ERA website.
For this project, I have calibrated the GR4J hydrological model using ERA5 and NRFA data for 314 river basin in the UK. Then I used hydrological model for generating runoff response to such an extreme rainfall scenario mentioned in “One Hundred Years of Solitude” of Gabriel Garcia Marquez: “The rains last for four years, eleven months, and two days.” Even if we try to solve fictitious questions like that, we can obtain reasonable answers. So one of the possible answers of the presented study is “Scotland.” To Scotland, you have to move to escape an exceptional drought which is caused by average rain lasting for 50 months. For further details, please follow Macondo in a trouble website.
The tiniest thing I have made – lightweight viewer for NetCDF files. If you have a NetCDF file with results you want to communicate – take a chance to WAT!
There were three categories to compete: Hack-Vis, Hack-Data, and Hack-Out. I submitted Beat of the ERA for Hack-Out, Macondo in a trouble for Hack-Data, and loose in both. Thank God I didn’t submit WAT viewer for Hack-Vis – it would be the third fail :-)
- The UK is strange and different but in an excellent way!
- The ECMWF is amazing
- Everyone who has participated in the Hackathon was indeed open for any talk
- It was a good experience, waiting for the next year Hackathon
- Learning from failures is hard
Many thanks to my project for funding – it was interesting to know that I can spend some money on traveling purposes without asking (impossible to imagine that in Russia).