Debugging in a Remote World
To learn more about Rookout go to softwareengineeringdaily.com/rookout.
As long as there has been software, there has been a need to maintain and debug it; and as the world we live in is changing, so does the way we debug software. One of the key changes the world is seeing is remote work.
Working remotely, and hence debugging remotely, isn’t new but it’s now taking the world by storm — whether we want it (modern workflows) or not (as elements like COVID-19 reshape our society).
The debugging relations matrix
Like most things in life this new way of working comes with both pros and cons, to best understand these and how this increasing trend will affect developers and the people around them, we should look at the debugging relations matrix, in which “Dev” stands for developers – e.g. software engineers, production engineers, SREs, etc. “Code” stands for the software being developed, used, and debugged. “User” stands for the person or persons using the software or overall product, and “Support” stands for the initial staff responding to detected issues or user requests.
- User / Code – Users can’t help:
With the complexity of modern software, migration to the cloud, and the overall expansion of backend components, users have very little access to the software and very little ability to understand it. While this isn’t affected directly by remote work; it does affect the other axes. - Dev / Code – Everything is remote and going further, companies adopting tools to adapt:
With SaaS and cloud, code in production is always remote, and often so are staging and even the baseline development environments. Similar is the case for IoT, and edge computing solutions. And these of course join the classic remote customer on-premise deployments. On top of this remote-work makes it impossible/impractical for developers to get closer to the software even in the cases where it was technically feasible. This explosion outwards means that all software is created equal, and organizations know it makes sense to invest in remote access and monitoring to the software- and budgets for monitoring / logging are increasing and access for debugging is becoming commonplace.
- Dev / User – “PEBKAC”:
Developers (on average) often find customer interactions challenging, coming from often introverted and sarcastic/cynical mindsets. A common saying among devs is – PEBKAC – problem exists between keyboard and chair. Working remotely makes all cases here much harder- both identifying it as true, or disproving it.
- Support / User – Getting to the bottom line / root-cause becomes harder:
Person to person communication suffers greatly from remote interactions, and this manifests here most dramatically, as both sides are remote from the code and understanding it. While this axis usually has the most channels set in place (chats, conference calls, customer relation management, case management), getting the full picture, and being able to recreate incidents on the customer side, or replicating incidents on the support side remains and becomes even more difficult. The pain-points of this axis stream down (rather than trickle down) to the dev/support axis. - Dev / Support – innovation is coming :
This realm of communication is ripe for innovation:- The need to allow remote access democratizes access to software and debugging- paving the road for new shared tools.
- Unlike the Support/User channel which has seen multiple new offerings, communication between dev and support remains much as it did a decade ago. Mainly using ticketing systems (e.g. Jira) with the slight exception of tasks/project management solutions (e.g. Monday) which aren’t really geared towards the needs here.
The conversation around debugging, replicating issues, and obtaining and observing the right points of data – was always complex and has become even more so with the explosion of software and going remote.
- Support / Code – Opportunity to empower support eng. – challenge in uplifting them:
While tier-3 or tier-4 support engineers are capable of doing debugging, their limited access to the software remotely or otherwise makes it impractical for them to attempt even in the cases where they have enough skill. Support’s limited access for debugging also affects their ability to communicate with developers, as both sides are looking through different lenses on the same issues, and fail to work on a joint status picture.
As remote work democratizes access (everyone is as far or close to the data as anyone else), by creating debugging solutions that adapt for both remote work, and various skill sets, there is a huge opportunity here to empower support engineers (on all levels) both enabling them to debug and solve issues on their own (removing pressure from R&D) and to better communicate with their fellow engineers.
How to debug remotely – a modern balancing act
Looking at the intersection of developers, support, code, and users, we’ve seen that there are many challenges that arise from the combination of the explosion of software and the growing trend of remote work. There’s no avoiding the question of what we can do to improve our debugging capabilities in this remote world. And for the best results we are required to balance bridging the gaps that are created by the new shift, while embracing the new mindsets that benefit from it.
Scale vs Familiarity
One of the main tradeoffs in moving to remote work is one of scale vs familiarity – while we lose the intimacy and closeness to both our code (few instances) and our customers (physically remote) we build and gain command and control solutions to manage things from afar and in greater scale. As such the first step should be to acknowledge this change and harness the power of tools and solutions that either have a scale mindset built into them, or empower our teams to adopt the scale mindset themselves.
A classic example would be to adopt microservices and distributed software principals, which at their core allow teams to break down software into smaller components, so they can be more easily deployed, scaled-up, but also debugged – by allowing multiple team-members to connect to multiple shared components, without having to synchronize the shared access.
Another example can be seen in a tool like Rookout, where scale and distribution is built into the fundamentals of the tool – flipping usual debugging flows on their head, and instead of having the developer/debugger connect (over the network) to each and every component they wish to debug, have all the components connect back to a command and control system, which allows developers to on-demand choose which components they wish to debug in an elastic system, without having to struggle with connections.
A Team effort – support and engineering
Another key area that is highlighted by the debugging relations matrix is team communication, and specifically communication between support and R&D teams. As communication suffers we can bridge the gap by investing more in culture, methods, and tools to augment communications. These should surpass shallow ticketing approaches, and focus on conveying quality data – rich with details on incidents, software, and user behaviours. By enabling our team members to share live detailed full pictures, and specifically live debug sessions so they can handoff investigations with ease.
Summary
As the explosion of software collides with a world adopting more and more remote work, many challenges arise for developing, maintaining, and debugging software. The effects can be seen all across the debugging relations matrix- from the developers themselves, the users, through the support teams, and of course the software itself.
Teams are now required to walk a fine line – balancing between the bridging the gaps created by the shift (e.g. gaps in communication, data access / availability), and embracing new mindsets that benefit from it (e.g. Distributed software and scale, remote access and control, shareable interfaces and sessions).
Indeed a challenging tightrope balancing act, but those who perfect it won’t only survive the shift, they will lead the future.
Find out more at softwareengineeringdaily.com/rookout.
