I have started organising my activities more strictly around the calendar, which means that my life just got a little boring, but a lot more organised. I can actually find that I can keep my sanity while doing the two dozen things that always seem to be on my mind. Occasionally, I slip up and find myself writing the post I meant to write the day before. But I am happy I got to writing this post this week.
First solving the problem
I read this post titled suffering-oriented programming earlier this week. It appears to be a trend now to call all the problems we face in software engineering as a variant of “oriented programming” or “driven development”. Anyway, the author here makes the case for letting suffering be a driver for improving your code. It is what could be called premature optimisation but on a broader scope. The author talks about the value of building proofs of concept and MVPs, essentially, before jumping on to focus on polishing it. It kind of reminds me about this xkcd comic.
Solving the right problem
In what makes a good goal, Leeor Engel writes about the value of solving the right problem. They talk about taking a step back to look at a problem differently. More specifically, they talk about, with an example, how changing the problem statement leads to a different and possibly better solution. The post talks about good goals and how to frame a problem statement that leads you to it. Like the article says: “How we formulate problems is just as important as how we solve them.”
School and grades
In the lesson to unlearn, Paul Graham talks about the education system typically seen in schools–specifically the idea of grading. This essay talks about grades in school and how that sets up people on a life long chase for finding the shortcut, or a way to hack the test. He relates this to how he sees the effect of the societal pressure of getting good grades later in life when founders hunt for VC.
The four horsemen of a relationship
In my last week’s post, I mentioned a podcast called Brain Science and a specific episode. This week, I listened to it again and paid more attention to a part about the four horsemen of a relationship. I tried to find out more about it and found a post where the four horsemen–criticism, contempt, defensiveness, and stonewalling are described in more detail with specific examples of how these are a sign that a relationship is on a critical path to failure.
On Maris and Vera
I found an in-depth article on The Atlantic talking about entirely unseen characters like Maris in Frasier. Of course, Maris never being seen in Frasier was little more than an interesting tidbit of the show to me. After reading the article, it dawned on me that it was weird that a character that drives so much of the story forward was not seen at all (except a shadow). It’s not the same as Vera on Cheers or other examples the article mentions. They are an amusing joke in their series but Maris is an integral and disruptive part of the story. I do see how keeping the character off-screen led to all the jokes that simply wouldn’t have been possible otherwise for so long. Look at Lilith, they couldn’t keep the jokes on her running for more than two seasons, and rightfully so.
DrupalQA image and PHP 7.4
I released the PHP 7.4 version of my image which packages some of the popular tools for running quality checks and tests for PHP applications. I later found that some of the builds on this version were failing. It was a problem with the Drupal code-sniffs which are a part of the Drupal coder module and I set about searching for the issue and found that it was fixed. When I looked at how come my image didn’t pick it up, I realised I was packaging an out of date release. I immediately released an updated image which contained the current 3.x version of the coder module. The builds still failed but that is because PHPMD has not made its way in the PHP 7.4 version of the image yet (as of this writing). As it makes sense, those builds are now using PHP 7.3 version of my image.
Drupal on Docker
At work, I am trying to streamline running tests on Drupal websites in a CI pipeline. While researching, I found an official Drupal Docker image which I had forgotten about. I found it when I was looking at juampynr’s image for similar purposes. I was not entirely happy with this and began to write my own image replicating what the official Drupal image did, except adding the entire Drupal installation to it (which I won’t need at all). With this, I was able to keep the image size as minimum as possible. The image works right now in my pipeline but I need to document it better and maybe even write a blog post documenting how to use it.
There are a lot more learnings here as to why specific aspects of our pipeline didn’t work such as how environment variables aren’t passed to Drupal code (that’s because Apache doesn’t pass any environment variables to PHP unless we explicitly say so). I managed to get it working with the results of my search implemented in the Dockerfile itself or the pipeline.
One of the things that stumped me throughout implementing this was how this image could be used as a regular image in which we run specific commands to test if it also has to run Apache in the foreground. The answer was obvious. The base image might want to run Apache in the foreground but we don’t. I had to start it explicitly in the background as part of the CI pipeline. I was confused because I didn’t see anything similar in the other image I was using as a reference. But then I found it in one of the Robo files and the confusion was cleared up. Things started to make sense again!
Database as an image
Another part of the CI pipeline is the database import, or more specifically, not having to import the database. The technique I am following is to load the database in an image and build tooling around it. Right now, I am just focused on getting it to work efficiently. Again, I looked at Lullabot’s image (by juampynr) and PreviousNext’s solution and tried to adapt to what I could. Specifically, I wanted to be able to use these images with Lando as well and I couldn’t use the same images as those two references. I like writing Docker images now. 🙂
Since Lando uses Bitnami images for MariaDB, I started with that and hoped it would work with minimum changes to Lando configuration. I was able to do just that with a very simple and elegant Dockerfile. It took a bit of reverse engineering how Bitnami constructed its images for MariaDB (their documentation doesn’t mention building on their images, just using them) and figuring out how to get my SQL imported at the right time. It turned out that Bitnami has support for initialising the database in their scripts. I just had to place my SQL file in the right location and run a specific sequence of scripts. The best part: it works with Lando with just two additional lines in Lando configuration.
I don’t have the Dockerfile code up anywhere right now but I’ll try and write more about it next week.
Clearing cache
I found that the database I was using had lots of junk in the cache tables. I used a stored procedure script to remove that, which should have reduced the size greatly (it reduced the SQL file size by ~30%). But the resulting image size didn’t change. I think that’s because MySQL pads the data files anyway. The compressed image should be much smaller anyway. (It is smaller. It is around 110 MiB as compared to 800 MiB uncompressed.)
Database sanitization
The solution by PreviousNext also sanitizes content in databases. I found that the efforts on the tool have split up in another repository. The previous solution was implemented as a Kubernetes operator, which I think is an overkill for my needs. I am leaving this problem for later once I complete the basic tooling for creating images out of databases.
Drupal errors
In all this testing, I faced weird errors. Even when cold-starting all the containers for a Drupal website (think Redis cache), Drupal website crashed with various types of exceptions. A cache rebuild would complete successfully but none of the pages would work. I first thought it was the invalid cache entries in the database, even if it was not used, but I removed all the cache entries as well. I had to get into Redis CLI and run FLUSHALL followed by a cache rebuild to get the site to work. I am not sure why but I think it has to do with how Lando runs health checks when the containers are all starting up. Maybe the health checks so early in the application cycle somehow corrupted the cache.
That’s it for this week. Thanks for reading this (you made it this far?) I hope to continue to keep writing this every week.
Leave a Reply