2021-01-13 - project infrastructure

there are 2 major elements for setting up infrastructure for a project:

1. CI/CD pipeline
2. SDK for development

it is still often to see 1st being non-existing or skeleton and 2nd in what i call “IaB” (infrastructure as binary ;)) – i.e. binary blogs just committed alongside with the code. typically this involves: 3rd party libraries, compilers, binary test data (like test images and test sets). when you see a project that's git repo takes 15GB, you know you are in that situation.

instead what i opt for IaC approach. then you can keep scripts, that can setup everything for you, with just a couple lines of code, keeping all the big, binary things away from repository.

for CI/CD this usually means CI/CD configuration (i.e. what steps CI/CD should execute to complete its work – think for eg.: Jenkinsfile or a Bamboospec) and CI agents setup (eg. with Ansible). this is not uncommon to be done like this nowadays, but still not all the projects use this approach.

setting up SDK in a IaC fashion however is far less common. most projects just do it manually (both on CI agents and developers PC). aside from brain-dead work it requires to setup it also causes issues (eg.: “it does not work on CI but worked on my PC!” syndrome, or versions mismatch between different devs' PCs). instead SDK should also be defined via IaC means. examples are Dockerfiles for docker images or Vagrantfiles for VMs. with such an approach you can be sure, that all devs use the same reference setup and that it is shared with CI as well.

but that's not all! there is also one more great, yet hidden benefit. with SDK defined in IaC, neither developers nor CI/CD need to be tied to a singe version of a compiler or library. instead, one can switch between different environments as easy as simply starting new docker container or a VM! this is great for maintenance, as now ppl can be fully productive on trunk/master branch and rapidly switch back to LTS version of the software, if a new bug is reported.

last but not least, this also keeps whole project's infrastructure aligned with an exact SW version / commit hash, as these files are just another code on the repository.