Didn't realise the Celtics needed so much

Been playing with heatmaps today. Got a inspired by the NBA draft and decided to scrape basic team stats from last season. Didn't realise the Celtics weren't up there on really any of the basic metrics last year.

The Celtics drafted shooting but seems like they could use a little bit of everything.

An updating map of Queensland air quality

Australians experienced pretty heinous air quality last summer. I was still being asked about it months later while backpacking through South East Asia. So, I've been wanting to build some maps, and start stockpiling data, ahead of the next fire season.

Luckily, the Queensland government has a lovely air quality feed. I threw together a quick map which should pull the latest data whenever it's updated. The red dots represent monitoring stations and the colour will get more intense the higher the readings of PM10 (a measure of particule pollution, notably emitted by car exhausts and fires). You can also hover over the stations for the exact numbers.

This project is also an excuse for something I've been wanting to try for a while - webscraping using Github workflows. I can't remember where exactly I came across the idea. But its pretty intuitive to scrape data straight into a csv in a Github repo. Even better: revision control.

I wrote a a Python script. A YML file instructs Github to run the script every hour. So far its working brilliantly and I can see a lot of potential in Github scraping.

Diving into the van Gogh letters

Vincent van Gogh's letters have been playing on my mind since I finished his biography a few days ago. Steven Naifeh used a close reading to paint a rich portrait, the ups and many downs. But what else can we see?

It's staggering how many have survived. Almost 1000 van Gogh-adjacent letters have been translated and put on a website attached to the Van Gogh Museum. 659 of these are letters from Vincent to Theo. 83 are to Vincent. The rest are a mixture of senders and recipients.

Obviously many letters have been lost, but right there you get a good sense of the relationship. Theo apparently saved hundreds of letters from Vincent. And this was before van Gogh was van Gogh.

Charting the letters across time gives you another clue as to the relationship. Vincent van Gogh only really focused on becoming an artist in the early 1880s, which is roughly when the letters to Theo really takeoff. This is also when Theo began to financially support him.

The sudden dropoff in 1886 and 87 is easily explained - those are two years when Vincent lived with Theo in Paris. There wouldn't have been many letters exchanged (just 19 survive).

You can see a similar trend in the length of the letters. The length picks up as Vincent repeatedly meets failure in his worklife and turns to Theo for support. There's another jump around the time Vincent takes up art in the early 1880s.

The average length is truly amazing when you consider the number of letters being exchanged. It reminded me how often Naifeh describes Vincent as copying out poems and bible verses when in a particular pique and needing to convince.

His exhortations eventually broadened to include love and belonging, melancholy and longing—subjects that clearly haunted him in his deepening alienation. So intense was his passion to persuade that letters alone could not contain it. By early 1875, he had bought an album for Theo and began filling its blank pages with long transcriptions from the works by these and other writers, all in a tiny, neat, error-free script. When he had filled every page of the first album, he bought another one and filled it, too, copying by gaslight late into the night.

But what about the content of the letters? They almost all begin with some version of "my dear Theo" and often end with a "your loving brother". But I'm not sure if that quite captures the relationship.

I ran a simple sentiment analysis algorithm on all the letters to Theo. It analyses the words and outputs "positivity" on a scale from 1 to -1. 1 being positive and -1 being mostly negative.

As you can see, apart from a couple of outliers Vincent's letters are mostly in the middle. Neither overly positive or negative. But also not what I would expect from letters to a close family member.

There's an obvious flaw here in that the letters would have originally been in either French or Dutch, so I'm working with a translation. But this roughly fits with the vibe of Naifeh's biography.

I think that might be all we'll wring out of the letters for now. I was thinking about looking into the distribution of words - find out the major topics. But I suspect there will be a lot of noise. Maybe let the art do the talking.

NBA All Stars are retiring older

With Vince Carter’s NBA career seemingly now over at 42 years young, I started to wonder whether NBA players are playing longer. There’ve been a fair few long-toothed retirees recently – Kobe Bryant was 37 and Dirk Nowitzki was 40 in their final seasons, for instance.

A couple more come to mind from earlier decades – Bob Cousy retiring (for a second time) at 41 in 1970 or Kareem Abdul-Jabbar at 41 in 1989. A quick scrape of basketball-reference.com suggests these legendary players are, in fact, outliers. But even so, All Stars, at least, have been retiring older.

Here’s a chart of all 503 All Stars listed on basketball reference. Their age in their final season against the season:

There’s limitation here. I didn’t feel like getting IP blocked for scraping every NBA player ever, so I decided to focus on a small subset – NBA All Stars. This is problematic for a number of reasons, noteably that All Stars are (probably?) more skilled and athletic than the average player, therefore more likely to play longer even if they do drop off. They probably also have longer, guaranteed contracts – meaning they stick around a few extra years regardless.

But as you can see from the following chart, the mean age of an All Star in their final season has increased in the past ~70 years.

So, the next obvious question is what’s going on? Some part of this is that athletes and the league are richer – NBA players aren’t likely to be working other jobs or have better offers elsewhere. Also medicine has advanced greatly and so some of what used to be career ending or degrading just isn’t anymore.

I wondered whether it might also be about better “load management”. That stars just play fewer, more strategic minutes per game nowadays, meaning they have fewer injuries and put less of a toll on their bodies. But a quick scrape suggests that isn’t So. Career minutes per game seems about the same for guys retiring recently as it did six decades ago.

If anything it seems that All Stars are playing far more. There’re definitely playing more total games. Some of this is due to more games in seasons and playoffs as the league has expanded. Private charters have also become a thing etc.

But better gear, medicine and sports science must come into it somewhere. Players and teams have access to so many more resources than they ever did, such as conditioning coaches, dieticians, chefs, masseuses and other therapists. This reminds me of a brilliant article from a few years ago, delving into all that Lebron James and his team go through to help him recover between games. The techniques are full on and constantly evolving.

But while they show their careers are lengthening, none of this really captures the impact over a career, which players and positions survive longest, and how a big a drop off there is. Maybe that’s what I’ll stick my nose in next.