{"id":10647,"date":"2026-09-22T16:15:06","date_gmt":"2026-09-22T16:15:06","guid":{"rendered":"https:\/\/djangostars.com\/blog\/?p=10647"},"modified":"2026-09-22T16:25:17","modified_gmt":"2026-09-22T16:25:17","slug":"web-data-portal-development","status":"publish","type":"post","link":"https:\/\/djangostars.com\/blog\/web-data-portal-development\/","title":{"rendered":"From Folders of Spreadsheets to a Web Data Portal"},"content":{"rendered":"

Someone asks your team for a subset of your data. A particular region, a particular set of years, a particular set of measurements.<\/span><\/p>\n

The request is reasonable. The data exists. But answering it means opening several files, remembering which version is current, checking whether those particular records can be shared, filtering by hand, and sending back a spreadsheet. It takes forty minutes if the person doing it knows the data well.<\/span><\/p>\n

And it is always the same person.<\/span><\/p>\n

Somewhere else, in a different kind of organization, a quarterly report is being assembled. The monitoring data goes back eight years. Every quarter, someone pulls the numbers together by hand, and every quarter it is built from scratch, because last time’s version does not fit this time’s question.<\/span><\/p>\n

This is the same problem. Not missing data, and not bad data. Data that exists is valuable and is reachable only by going through a human being.<\/span><\/p>\n

A note on why I am writing this.<\/b> I should say where I am standing while I write this. I did a PhD in economics before I moved into software, and I spent a lot of it on the wrong side of this exact problem\u2014pulling figures out of Excel files exported from several different platforms, none of which agreed on how a field should be named or what a missing value meant. The analysis was the small part. Getting the data into a state where analysis was possible was most of the work, and it was work nobody counted, funded, or saw.<\/span><\/p>\n

I am now building out this direction at DjangoStars\u2014data platforms for research and monitoring organizations\u2014and it is the part of our work I have made my own. That means the conversations before a project exists: working out what a group actually has, what they need it to do, and whether we are the right people for it at all. The platforms described further down are the work of our engineers rather than mine. My contribution is the problem, not the code: understanding it precisely enough that what gets built answers the real question instead of the one that was easiest to write into a brief.<\/span><\/p>\n

I talk to universities, research institutes, and organizations across Switzerland, Germany, Austria, and the Nordics about this every week, and what surprised me is how little it varies. Different countries, different disciplines, very different budgets\u2014the same folder of spreadsheets, and the same one person who knows where things are. Often the honest answer is that a group does not need a platform at all, and the sections below on when not to build one come out of those conversations as much as out of the ones that became projects.<\/span><\/p>\n

Why every data request lands on the same person<\/span><\/h2>\n

If your organization has been collecting data for more than a few years, you probably recognize some version of this:<\/span><\/p>\n