Solving Problems Using Code
Posted by Aly Sivji in Quick Hits
This canonical URL for this blog is alysivji.github.io. I have also set up nginx to forward requests to alysivji.com and sivji.com to this blog.
This is all well and good when are you running nginx out of a VM, but what if you are using the …
Posted by Aly Sivji in Quick Hits
Need to access a private Git repo inside of a Dockerfile during the docker build process.
The main idea is to use HTTP Basic Authentication while downloading files using the git command line client.
Create a Personal Access Token with repo scope using your Git provider's interface …
Posted by Aly Sivji in Quick Hits
Yesterday I gave an abridged version of my Data Science Workflows using Docker Containers talk at my employer's Biweekly Engineering Lunch session.
It had been a minute since I last gave this talk so I ran through the examples to make sure everything worked. Having all my examples containerized is …
Posted by Aly Sivji in Quick Hits
The paradox of choice in Python packaging tools makes dependency management more complex than it needs to be. In this Quick Hit, we will explore Python dependency management and demonstrate an easy-to-implement workflow to generate reproducible environments.
Note: this article assumes familiarity with virtual environments.
We use dependency management …
Posted by Aly Sivji in Quick Hits
I started my first tech job at a large corporation. We have lots of internal tools and systems that allow us to solve large-scale data problems. It's been an interesting change from startup life where platforms weren't as mature, where systems weren't as integrated into workflows and processes.
While I'm …
Posted by Aly Sivji in Quick Hits
I recently purchased a new computer and needed to transfer files from the old computer to the new one. Tried a USB drive, no luck. Tried copying files over the network via the macOS Finder, but the sheer number of files (~500,000) made this an exercise in futility.
Googling …
Posted by Aly Sivji in Quick Hits
Bandit is a static analysis tool that can find common security issues in Python code. The command line utility scans .py files and generates a report detailing issues by confidence and severity.
I ran Bandit on a few of my repositories and found I was using md5 to hash passwords …
Posted by Aly Sivji in Quick Hits
In July, I released my first open source project. It's an apispec plugin that generates OpenAPI Specification (aka Swagger docs) for Falcon web applications.
Apispec's design made it easy to extend core functionality for a specific use case. I extended the apispec.BasePlugin class, overrode a couple of methods, and …
Posted by Aly Sivji in Quick Hits
A couple of weeks ago, I stumbled across the Python Sorting HOW TO. Learned that Python can sort objects by multiple attributes using operator.attrgetter and operator.itemgetter.
In this Quick Hit, I will show the Pythonic way of sorting objects by multiple attributes.
Note: This example is leveraging Python 3.7 dataclasses for convenience. Objects can be of any type.
Posted by Aly Sivji in Quick Hits
Earlier this week, I went through the fantastic PyCon 2018 tutorial, Building a Search Engine with Python + Elasticsearch. This required me to have a local installation of Elasticsearch and Kibana.
Local installation? Time to spin up Docker, more specifically, Docker Compose.
Docker-Compose is a tool that allows us to define …
Posted by Aly Sivji in Quick Hits
Python 3.7 was released a few weeks back. Lots of new features in this minor release: data classes, new breakpoint keyword, dicts preserving insertion order, amongst others.
One feature that hasn't got as much press, or really any press, is an update to the unittest.mock module in the Standard Library. Starting in Python 3.7, instances of Mock
Posted by Aly Sivji in Quick Hits
Being a Software Engineer isn't just about being effective in a specific programming language, it's about being able to solve any given problem using the tools at hand.
This week at work I have to extend the functionality of a WordPress plug-in so it can fit into our microservices-based backend …
@ Python's Matrix Multiplication Operator
Posted by Aly Sivji in Quick Hits
2017 will forever be etched in our memories as the year Python overtook R to become the leading language for Data Science. There are many factors that play into this: Python's simple syntax, the fantastic PyData ecosystem, and of course buy-in from Python's BDFL.
PEP 465 introduced the @ infix operator that is designated to be used for matrix multiplication. The acceptance and implementation of this proposal in Python 3.5 was a signal to the scientific community that Python is taking its role as a numerical computation language very seriously.
Posted by Aly Sivji in Quick Hits
Testing is one of my favourite things about programming. There is nothing like the piece of mind that comes from being able to modify code without having to worry about breaking something.
Like most Python programmers, I started testing using the PyUnit framework (aka unittest) included in Python's Standard Library. PyUnit is part of the xUnit
Posted by Aly Sivji in Quick Hits
Dynamic typing refers to programs checking variable types at runtime versus compile time. By not having to define the variable type as we write code, we can work faster and take advantage of duck typing.
On the other hand, we do miss out on some benefits that types bring. Similiar to how the principles of test-driven development force us to write smaller, more testable functions, thinking about types improves software design by making us cognizant of the values our variables hold as well as the values our functions can return.
Posted by Aly Sivji in Quick Hits
In Python, functions are objects. This means we can return them from other functions.
In this Quick Hit, we will use this property of functions to mock out an external API with fake data that can be used to test our internal application logic.
Note: I previously used Python functions to simulate the behavior of a case statement
Posted by Aly Sivji in Quick Hits
I was floored when I discovered I could use tuples (and namedtuples) as Python dictionary keys.
Turns out this is not totally correct.
Started reading Fluent Python and learned that tuples are not always immutable. For instance, they can contain list elements...
Posted by Aly Sivji in Quick Hits
Even though my development folders are well organized, I still have to feel my way around the filesystem when I'm looking for a project directory. It's like trying to find a lightswitch in a dark room: a cd here, an ls there, maybe a find when I'm stuck.
I always …
Posted by Aly Sivji in Quick Hits
ChiPy hosts Python Project Night on the 3rd Thursday of every month. Groups of four are given an hour to work through a structured exercise with each person spending 15 minutes at the keyboard writing code. At the end of the hour, one group is selected to present their solution.
Posted by Aly Sivji in Quick Hits
A stack is a linear data structure that holds a collection of items. We can define operations performed in a stack from the point of view of a user:
.push() - add item to top.pop() - remove and return item from top.peek() - return top most element.is_empty() - True if stack …Posted by Aly Sivji in Quick Hits
Was reading the Python design FAQs and saw that immutable elements can be used as dictionary keys.
Wait a minute.... tuples are immutable, can they be used as dictionary keys? YES!