Raspberry Pi 4: How I built a twitterbot to track planes passing overhead
After getting my Raspberry Pi-based ADS-B receiver up and running earlier in the summer, I've been taking things a bit further with it. Some of the changes I've made have been hardware upgrades designed to improve both the range and reliability of the system, while others have been focused on finding uses for the data I'm collecting.
On the hardware side of things, as promised, I've swapped out the original whip antenna for a much larger 1090MHz antenna. While the antenna is technically designed for the top of a pole, I'm not planning on drilling holes in my roof, so it's instead duct-taped to the inside of a window at the back of my home, facing roughly north. It certainly has made a big difference, almost doubling range and improving reception for local traffic, including the helicopters up and down the river.
With a new antenna, range and accuracy is significantly improved, tracking aircraft nearly 200 miles away.
I've also taken advantage of recent changes in the Raspberry Pi 4 firmware to move away from running on an SD card and I'm instead using a 240GB M.2 SSD stick and a USB 3.0 adapter. By switching the Pi configuration to supporting USB boot, I've removed the risk of SD card failure (my original card wore out after just 3 months of operation, entailing a complete reinstall and reconfiguration) and also given the Pi a performance boost. Getting the SSD configured was the easiest part of the transition, as I found a script to handle cloning the drive for me.
Booting a 240GB SSD on a Raspberry Pi 4.
Finally, I switched from a simple fanless aluminium heatsink case to the more sophisticated Argon One case, which moves ports to a more sensible layout and adds a software-controlled fan alongside using its case as a built-in heatsink. It also offers a power button, which controls the power state of the device. As I'm running my Pi headless and feeding aircraft data to multiple ADS-B services, I've set the case's power-state jumper to automatically turn on if there's a power failure (and connected its PSU to a Wi-Fi-enabled power socket if I need to remotely restart the device if I can't use secure shell to login). I've liked this case so much, I've also used it for another Raspberry Pi that I use as a Linux desktop PC.
Developer
That all adds up to a much more powerful and more reliable ADS-B receiver. From a back window in London, I'm tracking aircraft nearly 200 nautical miles away and getting data for much of the helicopter traffic up and down the Thames low-flying corridor – both military and civil.
SEE: Technology in education: The latest products and trends (free PDF) (TechRepublic)
The next question was: is there more I can do with that data?
I'd seen that some folk were running various services on social media to report on local air traffic, and I wondered if it would be possible to do something similar. A search found a possible answer, a relatively old GitHub repository of someone's Twitter app that seemed to be designed to work with a setup like mine. The only thing was that it hadn't been updated in over two years, and some of the key components were no longer supported.
"Ah!" I thought, "Maybe I can update this."
So I cloned the GitHub repository and started work on updating the code. To be honest, I really didn't know much Python before I started working on the code, so some of my initial fixes were, to be honest, something on the hacky side of things. My first task was to get all the latest versions of the various libraries used onto my Pi. While the Python package manager Pip isn't installed by default with the Raspberry Pi version of Python 3, it's easy enough to install and then get the latest versions of the Selenium web-automation tool, the Pillow image-manipulation library, the Requests REST API handler, and a Python Twitter client.
The next issue was harder to deal with. The script I'd cloned from GitHub took a dependency on a generic webdriver client to handle automating screenshots on a headless browser. Unfortunately, that client, PhantomJS, was now deprecated, so I needed to find an alternative that would work on ARM hardware. While one option appeared to be to use Firefox in a headless mode, I couldn't get that to work. Instead I found a Raspberry Pi build of Chromium's chromedriver and installed that on my system. With a little work, I was able to convert the Selenium call that ran PhantomJS to take screenshots from the default Dump1090 aircraft mapping tool in the PiAware image to one that used Chromium. That removed the main incompatibility between the original script and modern software, and I was able to get the script posting screenshots and basic flight information to Twitter as aircraft passed over my home.
The next step was to find a way to translate some of the fight data into a human-readable form. As part of the ADS-B specification, all aircraft must transmit a unique identification 24-bit code that's usually displayed in hex form, which is also used by aircraft that are using simpler Mode-S transponders (often military or general aviation traffic that don't broadcast additional flight data). These codes are managed by the ICAO, the International Civil Aviation Organization, and are often referred to as ICAO hexes.
As the hex code is unique to the aircraft, it can be used to look up aircraft details in various databases, including type, registration, and operator information. I wanted to add this information to my bot's tweets, so I went looking for a public API that would allow me to get this data. Most are subscription services, with usage-based fees, and so weren't really suitable for a personal project.
However, I eventually came across a simple set of lookup APIs that I was able to add to my bot. I worked out where I need to extract the information and created a new Python function to call the appropriate APIs with the hex code received by my ADS-B receiver. It took a little work to understand how Python's Requests library worked with REST APIs, but I was soon able to convert hex codes and display results as part of a tweet. One side note: the free API I use doesn't have the depth of data that you'll get from a pay-for API, and some of the records are significantly out of date (some of British Airways' older A320 are still coded as being owned by British Midland, an airline that went out of business in 2012).
That put me in a position to push my cloned app to a new GitHub repository, where I could use the source control and remote editing tools in Visual Studio Code to secure shell into my Pi, and edit code on my development desktop before committing any changes directly onto GitHub. Having an editor that works directly on my code, with a terminal for quickly testing changes, and the tools for making commits to my repository has proven useful, along with its integration into my browser's dev tools.
Remote editing Python for my ADS-B Twitterbot on a Raspberry Pi using Visual Studio Code.
My next step was to change the mapping client from Dump1090 to the newer Tar1090, with its clearer aircraft icons and a new aircraft information sidebar. Installing Tar1090 from GitHub was easy enough, and I quickly had it up and running alongside the FlightAware fork of Dump1090 that's used in the PiAware distribution. It uses the same JSON data format for tracked aircraft as Dump1090, so I wouldn't need to write a new parser to extract aircraft details for tweets.
SEE: Programming languages: Developers reveal what they love and loathe, and what pays best
The web UI for the new map is very different, and so I had to modify the Selenium code to work with it. Using the F12 debugging tools in the Chromium-based Edge, I was able to narrow down the page elements needed for map interactions: resetting it between views, zooming in to an area around my antenna, and clicking on an aircraft to draw its flight trail and pop-up the aircraft details sidebar. All I needed then was to choose an appropriate resolution for the virtual browser and the size of the resulting screenshot.
#BAW2731 G-EUYR Airbus A320-232 British Airways: 0.4 mi away @ 2625 ft and 51.2° frm hrzn, heading W @ 202.6mi/h 16:57:14. #AbovePutney #ADSB #tar1090 pic.twitter.com/hJALNJ7n5A
— OverPutney ADS-B Bot (@OverPutney) October 18, 2020
A tweet and the associated image from my ADS-B twitterbot.
Much like this!
With that all done, my application is now happily posting away, sending the details of every aircraft that passes through the airspace over our portion of southwest London to Twitter. It's been an interesting project, and it's also nice to see that other people are finding my fork of the original ADS-B twitterbot useful, with the project quickly getting its own fork. I've even found myself having to give technical support. All that remains is to sit down and write some documentation to help prospective users get up and running, along with supporting another set of hex lookup APIs that offer free access to hobbyist ADS-B applications.