Solving Problems Using Code
As a learning exercise, I migrated all my side projects to Kubernetes. I was using Fluent Bit to forward application container logs to my ElasticSearch instance.
Everything was working as expected, but the cluster was starting to get expensive. DigitalOcean Kubernetes is competitively priced, but paying $80/month to run …
I started a new job in October and bought a 2018 MacBook Pro to celebrate. Purchased the 13" Touchbar with (mostly) maxed out components: 2.7GHz i7, 16GB RAM, and 1TB SSD. It's a huge step up from my late-2013 model; ~6x according to Moore's Law.
As soon as I …
A Python 3.6 feature that has flown under the radar is the revamped email package.
In this post we will explore email and walk through an example of creating and sending an email using the Python Standard Library.
emailThe email package is comprised of three major components …
I love using the Command Line; it's my favourite part of programming. It wasn't always this way. I used to worry that the commands I entered would destory the delicate equilibrium where everything just worked. Would this be the pip install that ended it all?
Enter virtual environments.
A virtual …
Summary
- Explore Plotly's new Dash library
- Discuss how to structure Dash apps using MVC
- Build interactive dashboard to display historical soccer results
I spent a good portion of 2014-15 learning JavaScript to create interactive, web-based dashboards for a work project. I wrapped D3.js with Angular directives to create modular …
Summary¶
- Explore
withstatements and the context manager protocol- Implement context manager class to query MongoDB
- Convert
try...finallyblock towithblock and increase code readability
I recently read Steve McConnell's Code Complete to level up my software development skill-set. The book has helped me become more deliberate about programming and problem solving in general. Before I sit down to write a single line of code, I take some time to plan out the work I am going to do versus code by the seat of my pants
(Note: This post is part of my reddit-scraper series)
Summary
- Running Scrapy spider as a script
- Scheduling script to run on PythonAnywhere cloud
Previously on Siv Scripts, we created a web scraping pipeline to pull Top Posts from Reddit and store them in a MongoDB collection. At this stage, we …
(Note: This post is part of my reddit-scraper series)
Summary
- Web frameworks intro
- Explore the Flask microframework
- Understand the Model-View-Controller (MVC) pattern
- Build Flask web app
Last time we started our web application adventure by learning how to generate dynamic HTML webpages from data stored in MongoDB using MongoEngine and …
(Note: This post is part of my reddit-scraper series)
Summary
- Overview of MongoDB
- Discussion of Object-Relational Mapping (ORM)
- Use MongoEngine to get items out of MongoDB
- Render HTML pages using Jinja2
- Interact with REST API to send emails with Requests
Previously on Siv Scripts, we implemented a web scraping pipeline …
Summary
- Discuss advantages of using Scrapy framework
- Create Reddit spider and scrape top posts from list of subreddits
- Implement Scrapy pipeline to send scraped data into MongoDB
Wouldn't it be great if every website had a free API we could poll to get the data we wanted?
Sure, we could …