whatupmiked intersection between networking, not working, and software


Autobots Transform

A few months back I finished a web-scraping program for downloading files. But I had not finished putting together many notification methods for when a file actually gets downloaded.

Last weekend I decided to scratch that itch by making a basic slack bot, slack web-hook, and twitter bot.

This ended up being alot easier than I expected because: - There are many modules for Slack/Twitter integration. - It is easy to find tutorials online.

So, rather than remaking the wheel let’s talk about a few of the different methods for interacting with the applications and creating a program.

Pattern of Requests

Both slack and twitter have documented public APIs that are readable. Using them follows this basic structure: 1. Create an authentication token associated with your account.
2. Store your token/credentials either in the program or as local evironment variables. 3. Retreive content with an HTTP GET to the associated API. 4. Modify or interact with the service using an HTTP POST. 5. Parse the return output and do something with it.

When you are using a python module like slackclient or twitter you interact with a class that is wrapping the API for you. This is convinient because they will deliver the content you want without you having to do the work of constructing the URI and unwrapping the payload. The trade off is that the module may not have all of the information you need.

Using the API directly is more flexible, but you have to implement all of the moving parts. That’s not a bad thing if you will be working with it regularly and intimately, but for a quick hack it’s probably not worth it.

Streaming vs. normal api and also webhooks.

A web hook is a custom listener that is attached to the platform. Your application pushes information to the web hook. webhook

This basic slack webhook is just pushing a notification from the application to your channel.

Whereas with a normal API, that application listens to the platform and reacts based on that information:


The twitter streaming api is essentially an active connection to the platform that is constantly notifying the application about whatever you are querying. I’m using this to parse the direct messages recieved by my bot. This challenge here is that the direct message API returns URLs that are going through the twitter url shortener. The expanded URL is also a redirect through twitter. So, to extract the actual URL of the message I needed to do a 2nd request to the actual message and then extract the real URL.