Notes from a technical leader

It’s a marathon, not a sprint

2021-03-21T09:00:00+00:00

Speed is a key feature of successful development teams and organizations. As software developers, it is our job to turn abstract visions and ideas into something that users can touch, feel and use, the faster the better.

The faster we can go from idea to working software, the faster we can earn revenue on those ideas and the faster we can gather feedback from actual users to use as input to feed the next set of ideas (or refinements of existing ideas) to implement. Speed also allows us to react rapidly to opportunities and threats in our organization’s surrounding environment, such as those coming from competition, regulations, customers and market trends. Additionally, speed means that we can address internal issues such as bugs and other problems in our systems in a timely manner.

Ok, so faster is better. Got it.

Well, not so, hmm… fast. When it comes to software development, there is another force coming into play as well. Being fast over a week or two, or maybe even a couple of months, is of relatively little use to our business if we can’t sustain that pace in the long run. Very few companies build their success based on just a few weeks of produced output. Therefore, the speed we are talking about here needs to be of the kind that we as a team can maintain over a long period of time. To use a running analogy, we are participating in an ultra-marathon race, not a 100 meter sprint. And, just as running as fast as you possibly can for the first kilometer of an ultra-marathon will likely exhaust you and have a negative impact on your overall race time, optimizing for short-term speed in software development will actually make you slower in the long run and put your business at risk of being outperformed by competitors using a strategy more suited to the distance.

This means we need to balance short-term speed against long-term and in most cases optimize for speed of the sustainable, long-term, kind. So while there certainly are cases when crunching or taking on technical debt in the name of short-term speed is warranted, for example to meet a critical deadline, that should be the exception and not the rule.

Let’s take an example. As a developer, the faster you can crunch out a piece of code, the faster you can get the feature that you are working on into the hands of your users. Which hopefully translates into more revenue for your business. All good so far. But when optimizing for pure short-term speed, you probably didn’t have time to think about how your change impacts the overall architecture or write those automated tests. Perhaps you took some shortcuts, ignoring quite a few ugly parts in the implementation or maybe did some quick copy-paste from other places around the code base since you didn’t feel you had the time to extract those common utility methods. This is short-term speed in its essence.

The paradox here is that by cranking out the implementation as quickly as you possible could, you actually made yourself (as well as your team and your organization) slower in the future. Why is that? This added “slowness” comes from two main sources. First, taking shortcuts when implementing a feature usually leads to more problems and bugs connected to that feature. This means that developers need to spend more time fixing bugs instead of adding value to the product. Most of the time, bugs and other issues also means that developers have to context switch to a much larger degree, which in turn kills productivity, adding to the slowness even further.

Secondly, the next time you need to make a change in that specific area, this will be harder and take more time due to the shortcuts taken the first time. In this case, you can either chose to clean up the mess introduce by the original quick n’ dirty solution or to shoehorn the change into the existing structure. Choosing the first option will take more time right now, but increase your long-term speed going forward. Going for the latter will obviously decrease your future speed even further, often creating a vicious circle.

To summarize, in order to achieve sustainable speed we need to focus on quality, which makes the process of writing that piece of code in the first place a bit slower. However, that lost time is made up for by being able to maintain a higher speed in the long run. So if we aim to optimze our speed over months or even years, which is the relevant timeframe for most companies, this is clearly a winning strategy.

So, what are the enablers for sustainable speed? That’s probably a topic for another blog post, but at a high level, it is pretty much all of those best practices in software development that are all too often ignored for the sake of short-term speed:

Test automation and testing tools
Spending time on design and architecture
Paying off technical debt
Continuous refactoring and improvement
Code quality
Process automation
Documentation
Communication

In closing, it is worth mentioning that the line between short and long-term is not always that clear-cut and that it is all too easy to get stuck in the wrong quadrant of the urgent vs. important matrix in our day-to-day work. When this happens, it helps tremendously to establish a long-term vision for which type of development organization to build and which practices, values and principles to follow based on that. In doing that, we can easily sanity check whether we are on the right track towards our vision, even though we of course sometimes need to make a few 100 meter sprints in the name of pragmatism to meet that critical deadline.

Lessons learned from 1:1s

2021-03-21T08:00:00+00:00

1:1s can be the most dreaded meeting of the week, both for managers and employees. But they are also one of the most important tools in a manager’s toolbox. Properly and consistently executed, 1:1s hold the keys for going from decent to great relationships with your reports. And having a good working relationship with your team members truly makes your life so much easier as a manager. It allows you to catch potential issues before they become real concerns. To build a healthy feedback culture where you both give and receive honest feedback. Address performance issues in a constructive way. Understand what motivates and drives your employees so that you can assign tasks accordingly. It’s an opportunity to explain and give context, resolve conflicts and have crucial career conversations. Basically, all the things that will allow your team to run as smoothly as a well-oiled machine.

And even though you will most likely spend a fair amount of your precious time in these meetings, experience tells me that it is time well spent. In the long run, I think it is actually more likely to save you some time. So I guess what I am trying to say here is: it’s well worth the effort to make good use of your 1:1s.

There are lots of great articles that cover the basics of 1:1s (for example here, here and here), so I won’t repeat that here. However, there are many more details once you go beyond the basics. As a manager, I think it is critical to adopt and adjust to your particular context in order to really maximize the value of your 1:1s. If you’re not sure how or where to begin, start with something as simple as possible based on one of the starter guides. Once you’ve done a few and are starting to get the hang of it, take some time to reflect on what works and where you’ve been struggling. Then adjust as you see fit. Rinse and repeat.

These are a few things that I’ve learned along the way and have worked well for me.

Rule number 1: It’s not a status update. Why? Because the main purpose of 1:1s is to build good relationships with your team members and trying to achieve that via status updates is simply not a very good way. I think we can all agree that status updates are not especially engaging and they risk putting the employee in a defensive position. In other words, not a particularly good starting point for building a great relationship. Status updates are of course necessary in many cases, but 1:1s are not the right place.
Be clear that it’s the employee’s agenda and that you expect them to come at least somewhat prepared. For many ICs, especially more junior ones, owning a meeting agenda and coming prepared is not something they are necessarily used to. So it pays off to spend some time explaining the concept and your expectations early on. Be extra observant of this in the first few sessions and give feedback accordingly to get them on the right track early on. This does not mean that you as a manager are not allowed to bring up your own items. Just that your team member’s agenda takes precedence and is the one to start with.
Consistency is key. Some people need more time to build trust in you before they feel comfortable to open up and discuss the things that are truly important to them. This is ok. Even though the first couple of 1:1s may be painful, the best you can do is to endure and continue. It may take several weeks or even months before things improve and you get a good flow in the discussions, but at some point you will. I promise.
A good way for you as a manager to lead the way in building trust is to be open with your own doubts and struggles. Sharing your experiences, feelings and failures when relevant is an invite to the other person to do the same.
Consistency will be hard to achieve if you repeatedly cancel your 1:1s, and that also signals that the meeting is not very important to you. So try to avoid that at (almost) all cost. For the same reasons, rescheduling is not optimal either, but at least slightly better and I’ve found it hard to avoid in practice. Just try not to make it a habit and always reschedule to the same week.
Be present. Rushing straight from another meeting into a 1:1 is almost never a good starting point for a productive 1:1. This is also the reason why I prefer not to have too many 1:1s back-to-back. I simply find it too exhausting to be truly present for more than two, maximum three, sessions in a row and can literally see how my performance in the later sessions degrades significantly when I exceed that limit. The exact number may be different for you, but I think we all have that limit and it’s important to be aware of it and respect it.
One way to reset before or between sessions that has worked well for me is to make sure to get a few minutes alone before the 1:1 starts. Close your eyes, focus on your breathing and try to let go of all the other things that bounce around in your head is something that has worked wonders for me.
Be curious. Ask open ended questions. “What’s on your mind?” is a great starter question. “What conclusions do you draw from that?” is a really good question to follow up with. The cool thing about both of these questions is that they can be repeated over and over again, almost indefinitely, and get you one step closer to the “real” issue or insight on each iteration. The one place where the DRY principle does not hold, I guess. Avoid “Why did you…” type of questions as it tends to put the employee on the defense. Such questions can often be rephrased as a “What…” question instead, which is a better way to frame it in such cases. Other questions that can unlock a 1:1:
- “What’s the one thing you would change in our way of working?”
- “What’s your main challenge right now?”
- “What mistakes do you think we’re making?”
- “What has been the best/worst that has happened since we last spoke?”
- “What did you think of X?”
- “What’s your main productivity blocker?”.
Don’t be afraid of silence. In many cases, silence is your friend, especially the slightly uncomfortable one. It means you most likely asked a good question that made the other person stop and think. Don’t ruin the moment by answering your own question, give the other person an easy way out or rush to the next one.
Don’t save feedback until the 1:1. Always aim to give feedback as close to the event as possible. However, 1:1s are a good time to follow up, discuss and track progress on previous feedback items.
Embrace the uncomfortable. If your 1:1s are not a bit uncomfortable at times or if you’re not sometimes slightly nervous about bringing up an item, then you’re likely dodging certain topics and not getting into the really crucial conversations. Which is unfortunate since you’re then missing out on the main benefits of 1:1s. So treat overly comfortable and smooth 1:1s as a code smell, it may be a sign that you’re doing something wrong.
Take notes, preferably on paper. Yes, I know. We live in the digital age and taking notes on paper feels a bit ancient. Plus, it’s really a pain to transfer to digital form, which you likely may want to do to keep your notes organized and searchable. But the truth is that opening up your laptop during your 1:1 doesn’t really signal that you care and that you listen. For all the person on the other side knows, you might just as well be browsing through your emails or responding to your latest Slack messages (which can be tempting also). And this is certainly not the message you want to convey in a meeting that is all about listening, caring, building trust and having deep conversations.
Do you need to take notes at all? Well, you might be a mastermind champion, but at least I have problems remembering what I did yesterday. Keeping notes of each 1:1 has been very helpful for me when following up on topics that were discussed in previous 1:1s or to spot trends over time the conversations that could warrant a conversation on its own.
Individuals are different and I think it is good to adjust the exact format to suit the person you are having the 1:1 with. Some guides tell you to stick to a strict agenda. That has not worked very well for me and nowadays I’ve come to believe that it is perfectly ok to allow the format to be a bit different depending on what works best for the employee. As long as you don’t make it a status update (rule number 1, remember?) and let your report own the agenda (rule 2), you should be fine.

Three flavors of tech leads

2020-09-28T16:24:01+00:00

During recent years, I’ve come across (and worked with) three basic variations of the Tech Lead role. While the essence of the role remains relatively constant, each flavor adds some interesting characteristics and comes with its own pros and cons.

Project based

The project based Tech Lead is not a point on the career ladder, but rather a role that any engineer can take on from time to time for a single project. As such, it comes with a well defined start and end date as well as with a clear goal of ensuring the technical quality of the project’s delivery. Unless the project is large, this flavor usually involves less focus on architecture and more focus on the project management aspects. The ability to prioritize, break down large tasks into smaller chunks of work and communicate with non-technical stakeholders are often key skills to either have or develop in this role.

It’s a great opportunity to introduce less experienced developers to the role by letting them try it out during a limited period of time and gain relevant leadership experience without the long term commitment that a more permanent setup might imply. It is also possible to tune the complexity by offering different project sizes depending on the engineer’s current level. However, you might lose some of the long-term thinking and influence that the other variations of the role usually include.

Component based

The component based Tech Lead is responsible for a technical component within the overall technical platform. Most commonly, the component here is either backend of frontend, but could also be sliced differently depending on what makes sense for the organization. It’s a more technically focused role where long-term thinking and planning is a large part. You get more influence over architecture compared to the project based flavor and often less focus on project management.

While this might seem like a fairly natural way to slice the role’s responsibilities, the downside is that it becomes very easy to focus too much on the technical aspects and lose the connection to the needs of the business. Another challenge for the component based Tech Lead is that it is difficult to take responsibility for a full product delivery or project since such initiatives often span multiple components. However, in certain scenarios, for example in the case of a very complex technical component, the positives might still outweigh the negatives.

Team based

The team based Tech Lead is responsible for the technical quality of deliveries within an entire software engineering team. Implicitly, this includes the technical components and the projects that the team is responsible for, so you might argue that this is a combination of the two previous ones. However, in addition this flavor also comes with more focus on leadership, management and “people” skills. It gives you lots of influence but also requires a broader spectrum of competencies. As such, previous experience of one of the other flavors might be a good stepping stone into this role.

The Tech Lead Role

2020-09-06T18:24:01+00:00

So, what is a Tech Lead and why do we need one? As with most roles within our field, there is a fairly wide spectrum of definitions. However, most people tend to agree that the role’s primary focus is to take responsibility for the technical quality of deliveries within a defined area and that it includes a mix of programming, architecture, project management and leadership. It is usually not a full-time management role, meaning that the Tech Lead is still expected to code at least some part of their time. As such, it is often seen as a good next step for someone with a few years of experience as an individual contributor, who now wants to take on more responsibility and dip their toes into leadership, though approaching it from a very technical angle.

For organizations with multiple development teams (or single-team organizations without technically strong C-level leadership), introducing this role can also be a good way to bridge the gap between senior technical management and the teams. By pushing technical decision making authority down to the tech leads in each individual team, the organization can maintain its agility as it grows and continue to make technical decisions that are relevant for each team, without having to resort to micro-management. This also makes teams more autonomous and independent, which is generally “A Good Thing” as it increases speed of development by removing external dependencies and unnecessary lead times. Of course, this only works if the senior technical leadership (e.g. the CTO and/or VPE) sets direction and alignment across all teams by communicating a strong vision, strategy and goals combined with a set of guiding principles to consider when taking technical decisions.

When crafting the role description for a Tech Lead, the key thing to keep in mind is that success in this role requires more than just technical skills. Therefore, it doesn’t always have to be assigned to the most experienced and/or technical engineer on the team. On the contrary, most reasonably good developers with at least a few years of experience should have the necessary coding skills for the role. A good understanding of technical best practices and when to apply them is obviously important as well as being able to reason about architectural topics. But even more essential (and harder to find) skills are the ability to communicate, coordinate and cooperate with both technical and non-technical people. And, the project management part of the role requires an aptitude for planning and breaking down complex projects into smaller tasks that can be delivered incrementally to minimize risk and maximize learning.

Since it is a role focused on excellence in delivery, it is also important to be able to understand and focus on producing business value and as part of that, be able to find the right balance between technical perfection and pragmatism. We certainly don’t want to take on a massive amount of technical debt in the solutions we produce, but we also don’t want to spend forever polishing them to perfection. Knowing where to be on this spectrum in every situation requires some experience. This is an area where more senior technical leaders should put extra focus through coaching and mentoring.

When it comes to anti-patterns, the most common one is to treat the Tech Lead role as a one-man show who takes all the important technical decisions, assigns the most interesting tasks to himself and acts as a gatekeeper when accepting pull requests in the team. Instead, a good Tech Lead should be able to leverage the team’s collective strengths, skills and competencies to drive results, acting more as a facilitator than a dictator. In this area, the Tech Lead’s leadership skills really come into play. You want to find someone who is helpful, humble and inclusive and whose ego does not get in the way of finding the best technical solution. Needless to say, these softer skills are often where inexperienced Tech Leads struggle the most and where good guidance and clear expectations from senior technical leadership is crucial.

Another area where people new in the role often struggle is around delegation. As an individual contributor, it is natural to measure your performance by the things that you deliver yourself. As a Tech Lead, your performance is instead measured by what the entire team delivers. Therefore, one key to success is to focus just as much on what the other team members are working on, and how you can help them, as on your individual deliveries. This shift in perspectives can be really hard for new Tech Leads and it is not uncommon to feel a lack of accomplishment during days when you have not produced many lines of code yourself. Of course, this does not correspond to reality (quite the opposite in fact), but years of getting used to the tangible and instant gratification of writing code as a measure of performance can be hard to unlearn. Again, good coaching and mentoring from senior technical leadership, who have often been through this themselves, is invaluable here.

Since it is a role with emphasis on much more than just technical expertise, there is some likelihood that people new to the role will realize that it is not their cup of tea after all and will want to get back to a pure software development role. This should be perfectly ok. As a leader, the best you can do is to make it very clear that this is not seen as a step back career-wise and that there is a rollback plan in place. If not, you will either get an unhappy and unmotivated employee or a resignation, and neither of those alternatives are very appealing.

To summarize, the Tech Lead role is a great stepping stone into leadership and a good way of providing scalability to a growing technical organization while at the same time offering more diverse growth opportunities for your employees. However, the responsibilities and focus areas of the role varies a lot from organization to organization. Therefore, it is important to think hard about what the role should look like in your context to match your organization’s needs. And to keep in mind that being a Tech Lead is about much more than just technology. Especially new Tech Leads will require lots of guidance and support to become fully proficient and comfortable in the role.

In an upcoming post, I’ll write more about different flavours of the role and how you can use those to offer more flexibility and tailor the role to your organization’s specific needs. Until then, I can highly recommend Patrick Kua’s blog where there is lots of really good content about the Tech Lead role as well as other aspects of technical leadership.

Stay tuned!

Onboarding starts earlier than you think

2020-05-09T14:56:01+00:00

Over the years, I’ve seen way too many engineers get off to a bad start during their onboarding process. And even though the evidence is very much anecdotal, I think it is fair to say that the onboarding phase is one of the riskiest parts of an employee’s lifecycle at a company. Get it wrong, and you risk losing your hard earned recruitment or have them create a mess in your team that you’ll spend months cleaning up afterwards. Get it right, and you’ll soon have a highly productive, motivated and loyal employee that boosts both the efficiency and morale of your team.

In my experience, onboarding can go wrong for two reasons. Either too little support, which makes the employee lose motivation and look for opportunities elsewhere. Or, you as a manager are too late in giving constructive feedback and correct unwanted behavior, creating a toxic situation later on. To be fair though, in the latter case you probably recruited the wrong person to begin with, since I’ve rarely seen cases where this was possible to correct. In such situations, it is crucial as a manager to take the hard but necessary decision to let such a person go as early as possible. Not a fun place to be in, but since I’ve never seen recruitment success rates beyond 80% (at least not at scale), it is most likely something you will be exposed to sooner rather than later so you’d better learn to spot such situations and handle them in the best possible way.

One very common mistake, that I’ve made on more than one occasion myself, is to think that the onboarding phase starts when your new recruit does her first day at the office. I think one of the keys to a successful onboarding experience is the realization that the process starts already during the recruitment process.

At this stage, you need to ensure that you give an accurate picture of your tech stack, the role, the team and the company to set reasonable expectations and avoid negative surprises for your recruit later on. No point in selling your tech stack as a new and shiny Tesla if it’s more like an old and rusty Volvo, as you’ll set things up for a very negative experience as soon as your recruit does her first git clone. Additionally, you’ll likely attract the wrong type of personality for your needs which you’ll regret later on.

From the candidate’s point of view, a recruitment process can be perceived as a very rapid affair. One day, you’re being contacted by a recruiter and persuaded into a meeting “just to hear more” and before you know it, you’re putting your signature on a contract and have landed a new job.

Once the contract is signed, it is not uncommon for your recruit to have second thoughts or be persuaded (by social pressure and/or compensation) by their current employer to not leave after all. While this is of course not something that you, as the new manager, can fully control, the impact is multiplied by radio silence from your side.

To mitigate this, make it a habit to always follow up with your new recruit as soon as possible after the contract is signed to check that their resignation has been followed through and to give her some moral boost by reinforcing that they made the right decision by telling how happy you are that she has decided to join your team and what a bright and exciting future you’ll have.

Another nice touch is to cc the new recruit in the company-wide email communication that is (hopefully) sent to inform about the new employee. If you have a decent company culture, this should yield at least a few encouraging replies to the new employee to further reinforce the decision.

Also make sure to give the hire a few status updates along the way around how things are going in your team and what is happening in the company. This will further strengthen the relationship and initiate the process of making the employee feel part of your company even before she has started.

Once the employee-to-be is a little bit more than half-way through her notice period, it’s a good time to meet up for an informal lunch or coffee. Take this opportunity to fill her in on what is going on in your part of the organization, describe projects that she will likely participate in and answer any questions she might have. This can also be a good time to provide some tips, suggest areas to read up on as well as some mentorship on how to approach the first couple of weeks/months in the organization.

Finally, a few days before the first day at the office, make sure to send a welcome email with details around when to show up and the schedule for the first couple of days.

Phew. That’s a lot of work, and your employee has not even started yet (a topic in itself, I’ll write about that some other time). But it pays off and increases the odds of a smoother ride once the new hire starts for real. And since the vast majority of companies out there are surprisingly bad in this area, even small and simple things can really make your company stand out.

Using data to guide refactoring efforts

2020-04-11T11:24:01+00:00

“This code base is a mess, we need to spend more time on refactoring”. Chances are, you’ve heard this complaint from developers in your team on more than one occasion. Probably even said it yourself a few times. And while most of us can sympathize with a statement like this, the analysis usually doesn’t go much deeper than that. Way too often, the next step is simply to put your noise cancelling headphones on, roll up your sleeves and dig into it, randomly refactoring pieces of code that happen to cross your path. I think we can do better than that.

Across our industry, there seems to be a fairly wide consensus that refactoring efforts should be integrated with regular development activities as opposed to being handled as separate stories on the backlog. Martin Fowler calls this opportunistic refactoring and Uncle Bob refers to it as following the boy-scout rule, but in essence this approach means that you let product priorities drive what parts of the code base you refactor. The payoff for a refactoring effort strongly depends on how often you will make a change in that area in the future and, in absence of a crystal ball, product priorities are a decent indicator of that.

This makes a lot of sense, but what it fails to tell you is how much time you should spend on refactoring as part of a single product feature. Gut feeling seems to be the most common approach. A slightly more structured tactic could be to agree on a fixed percentage as part of the total story estimate. However, both of these strategies can lead to refactoring efforts in places where it does not make the biggest impact. Which highlights the main challenge with refactoring: It’s really hard to know when to stop.

Refactoring is commonly an unbounded task with no clear acceptance criteria making it very easy to go down a rabbit hole where the refactoring effort takes a ridiculously long time compared to the actual feature that started it all. Adding to this, refactoring suffers from diminishing returns where the first 20% is likely way more valuable than the last 80%. The challenge, of course, is to know what constitutes those 20%.

Luckily, there are a few simple steps you can take to make this decision more data-driven.

Start by calculating how often the source files you want to refactor are modified. This can easily be done using your git log in combination with some command line wizardry:

git log --pretty=format: --name-only | sort | uniq -c | sort -rg

Next, grade the code quality of the files, either using a metric such as cyclomatic complexity or by assigning it a subjective score according to some scale (e.g. 1-5, good/average/awful, WTFs per minute, etc.).

Based on the code quality and change frequency, you can now plot this in a graph as illustrated below.

Divide the graph into four quadrants and let this guide how much time you spend on refactoring.

Bottom-right. This is where you want to spend most of your total refactoring effort as these are files that are changed a lot and have bad code quality, giving your investment in code quality a higher likelihood of paying off.
Bottom-left. Bad code quality, but not very frequently changing files justifies a medium refactoring effort. Why not low, you might ask? After all, these files are not touched very often. Well, this can of course be argued, but in my experience this category adds to the team’s overall cognitive load and reduces their ability to understand and reason about the system as a whole, even though the actual change frequency is low.
Top-right. Good code quality and high change frequency justifies some refactoring, but not that much compared to the previous two quadrants. Due to the diminishing returns of refactoring, the high-value 20% are probably already addressed making further efforts less valuable.
Top-left. Good code quality and low change frequency. These are the ones you can safely ignore from a refactoring point of view. By all means adhere to the boy-scout rule and make small adjustments, but don’t place your high stake refactoring bets in the area.

By using these simple heuristics, you can make your refactoring efforts more data driven and greatly increase the probability of spending your time where it has the most impact.

Getting up to speed on a new technical platform as a manager

2019-12-01T13:31:21+00:00

When joining a company as a technical manager, one of the major challenges in front of you is to learn the details of the technical platform(s) that you are responsible for. You are most likely not coding any more, or at least not making any major contributions to the code base, and with your days filled with meetings and other management duties, it can be a real challenge to find the time to actually dig into the more technical parts in order to get the understanding needed to do a good job.

Depending on your role, background and position on the company ladder, the actual level of technical detail that you need will vary a lot. So before you jump into things, spend some time analyzing what amount of technical understanding that you actually need to do your job well.

For example, do you need to understand all details in the code, is it enough to grasp the most critical parts or should you start to build your knowledge from the architectural level and upwards? As engineers, knowledge is our main currency and therefore our gut reaction is often to go deep and detailed. However, that’s not necessarily where you’ll spend the most part of your time as a leader. Also, think hard about what effort it will take to keep that understanding up to date over time and if that is a realistic effort to spend in the future.

Having been in this situation myself several times, here are a few things that worked well for me:

Draw a system map of all components in the platform and make sure you have a good understanding of the purpose of each part and how they interact. Make sure to ask questions to clarify the history and reason behind the not so obvious pieces. Surprisingly often, this is not documented at all and creating such a document will provide value both to future team discussions and to quickly give newcomers a good mental picture of your platform.
Produce technical documentation and presentations. This is likely something you will have to do as part of your role in any case and it is a great opportunity to get more familiar with the platform. Initially, you will need help from your senior engineers but eventually you’ll manage most of it yourself.
Ask your engineers where their biggest pains are in the technical platform and make sure you understand not just what but also why. This will give you an understanding of how things are connected, especially the not so shiny parts. As an added benefit, you’ll get a nice backlog of technical debt to address. Also, as these will likely be areas where various issues will surface in the future, you will then be in a better position to understand (and explain) why those issues happened.
Facilitate postmortems and retrospectives. Again, make sure you ask all the whys. Being the new guy or girl, it’s ok to ask lots of stupid questions, so make sure you’ll take advantage of that privilege.
Treat your team’s code review process as an opportunity to read and review code. Initially, you will focus mostly on just understanding the code and give feedback around general coding practices, but quite soon you will be able to have input on the actual solution as well.
While I think that you should mostly focus on leadership, I sometimes find it useful to take on some small development tasks, such as fixing a trivial bug or a small improvement outside of the iteration’s critical path. It will force you to dig around in the code base and familiarize yourself with surrounding processes such as CI, tooling and build processes.
Pair-program with the engineers in your team, both senior and junior. You probably will have to take more of a backseat role, but you can still learn a lot and it is a perfect opportunity to ask questions. And when doing this with more junior engineers, there is likely a ton of useful feedback you can give them right away around structure and clean code without even knowing the details of the code base.

Not your typical DevOps whitepaper

2019-11-10T15:47:40+00:00

In my opinion, one of the best, and perhaps also one of the most underappreciated, developments in our industry the last couple of years is the State of DevOps report and its companion book, Accelerate, that presents the findings and the research behind it in more detail.

I will not go into the details of the report, it is available online and you should really go and read it for yourself, but there are several things that stand out and makes it so much more than your typical DevOps whitepaper:

It is (as far as I can tell) backed by solid research. But unlike most research, it is still highly approachable, practical and applicable to the day-to-day operations of almost any software engineering organization out there.
It puts an end to the myth that speed and stability in software delivery are opposing forces. You can (and should) be great in both dimensions and this is why a DevOps culture is such a critical piece of the software delivery excellence puzzle.
In our industry, when debating why certain technical practices should be followed, our arguments have historically primarily been based on opinions/gut-feeling (at worst) or anecdotal experience (at best). This can lead to cargo cult behavior and difficulties in trying to convince non-engineering parts of our organizations that technical excellence is worth its (high) cost. With this research, we can finally focus on the things that are actually evidence based and place our technical investment bets accordingly.
By connecting software delivery performance to organizational performance, such as profitability, productivity and customer satisfaction, it gives you as a technical leader strong arguments for why investing in a decoupled architecture, clean code and solid (preferably automated) technical processes is so much more than something you should do to keep your engineers happy. It is actually critical to the success of your company and now you have the research to back it up.

So, once you’ve read the report (or the book), here’s an exercise for you as a technical leader:

Find and list all the key drivers of software delivery performance that are mentioned in the report.
Score your organization in each dimension. Or even better, let your team(s) score. Red/yellow/green, 1-5 or some other scale, doesn’t really matter. Now you know where to focus your efforts.
Prioritize one or two areas and dig deeper into those to really understand where the limiting factors are.
Work hard to improve in those areas and then score again in six months.
Repeat.

Congrats, you’re now on the path to becoming a high performing software delivery organization!

Consumer driven contracts

2019-10-12T17:17:04+00:00

Recently, I decided to dig a little deeper into consumer driven contracts and more specifically the Pact framework. In essence, consumer driven contracts are about having consumers of a service define their expectations (i.e. contract) on that service in terms of tests that can be executed on both the consumer and the service provider side. Such tests can then be run as part of the CI build of the service provider as well as the consumer. If done right, the provider service can evolve independently as long as it doesn’t break any of those tests and you therefore limit the need for extensive end-to-end testing as a means to catch breaking changes. Which is good since such tests can be both time-consuming and fragile. In this age of APIs and (micro)service oriented architectures, I think this is a very interesting approach to QA and test automation. And arguably also a fairly underutilized one.

So, let’s get started with an example. In this scenario, we are integrating with a service that can provide a list of users. In the client code, we call this service like this:

class UserApi():
  def __init__(self, host='http://127.0.0.1', port=5001):
    self.base_url = f'{host}:{port}'  

  def all_users(self):
    response = requests.get(f'{self.base_url}/users')
    return self._handle_response(response)

  def _handle_response(self, response):
    response.raise_for_status()
    return response.json()

  ...

A consumer driven contract test implemented in Pact could then look like this:

expected_user = {
    'name': Like('John Doe'),
    'id': Like('abc123'),
    'active': Like(True)
}

def test_get_users(self):
  expected = EachLike(self.expected_user)

  pact
    .upon_receiving('a request for users')
    .with_request('get', '/users')
    .will_respond_with(200, body=expected)

  with pact:
      result = user_api.all_users()

  self.assertEqual(result, get_generated_values(expected))

When we run this on the client side, Pact will start a mock http server that responds to a GET /users request with a 200 status and a list of users.

Once this passes on the client side, we can generate something called a “Pact file”, which is a a JSON representation of the test above. We then share this file with the service provider, either manually, using a shared file server (or an S3 bucket or similar) or by publishing it to a “Pact broker”, which is basically a repository of Pact files that both the client and the service provider have access to. The process of generating Pact files and publishing them is typically done as part of the client’s CI flow.

On the service provider side, the interactions defined in the Pact file is then executed against the service and verified according to the rules defined in the test. This is done by the Pact framework and only requires a running provider service. Note that since the Pact file is defined in JSON format, the client and service provider can be implemented in different languages. Pact currently support Python, PHP, .Net, Ruby, Java (actually all JVM compatible languages), JavaScript, Swift and Go.

Since the client (where the test is created) can’t control exactly what data that will be returned (would make it a very brittle test and make it challenging to include dynamic data like timestamps or ids), “minimal verification” is used in this phase. This is why we used the Like(...) construct when declaring the expected_user variable above and it basically means that Pact should only verify the data types and not the actual contents. This can also be done using regular expressions.

As a next test, we can make sure that the service responds with a 404 if we ask for a user that does not exist:

def test_get_unknown_user(self):
    pact
      .upon_receiving('a request for a non-existing user')
      .with_request('get', '/users/non-existing-id')
      .will_respond_with(404)

    with pact:
        self.assertRaises(HTTPError, user_api.user_by_id, 'non-existing-id')

Of course, this assumes that there is not actually a user with the id non-existing-id in the provider service.

So far, so good. As long as tests are as simple as one single interaction, things are very straightforward. However, if a test depends on something being setup in a certain way in the provider service, things become a little bit more complicated. As an example, let’s say that we want to test to get a user by id. To do that, we may first need to create a new user on the service provider side and get the id from there. Pact’s solution to this is something called “provider states”.

Provider states in Pact are setup as a special endpoint in the provider service (although it doesn’t strictly have to be the same service, it could also be a “test” service running side-by-side with the real provider service). This endpoint contains the setup code for different scenarios. The client can then trigger these setup endpoints by using an agreed naming scheme. This is a little bit cumbersome and introduces some tighter coupling between the client and the provider, but I guess it is a challenging problem to begin with. It is not recommended to use the public provider API to setup data (e.g. calling POST /users before GET users/xyz), so I get the impression that this is a conscious design decision from the Pact developers to make sure that tests don’t become too brittle.

Some things to keep in mind:

These tests should not verify functionality, only focus on testing the format of the requests and responses between a client and a provider.
The client should only verify the things (e.g. data fields, response codes etc.) that it actually needs, to give the provider as much freedom as possible to evolve.
This type of testing is not really suited for public APIs or in other cases where the client team and the provider team don’t communicate on a regular basis.

To summarize, even though I only scratched the surface this time, I think it was a pretty good experience and in my opinion consumer driven contracts should definitely be something to assess as part of your QA automation toolbox when working with micro-service oriented architectures.