Software, science, life...

Off payroll legislation in the private sector - the good points

Thu, 15 Jul 2021 00:00:00 +0100

Yup, that got your attention. Clickbait title aside, the changes to ITEPA 2003 Ch 10 introduced in the Finance Act 2020 does have some good points. Have your eyes glazed over yet? :)

Before I continue here are the usual disclaimers - the following is my opinion only - I could be wrong in my interpretation, and should not be construed as legal or tax advice. If you spot a mistake, feel free to correct it in the comments. I’m talking about contracting in the IT sector specifically as that is what I know. I have no idea if independent professionals and freelancers in other sectors have had similar experiences.

So what are you anyway?

I’m a software engineer and agile coach and I provide services through my limited company. I conduct my business as a proper business, I am most definitely not a temp or a client employee in all but name. That difference has been the crux of the intermediaries legislation (aka IR35 or ITEPA 2003 Ch 8).

Professionals in my situation have been navigating the contract and tax minefield around this perception of who we are for over 20 years now. It’s an utter PITA and takes up resources that we could be using to provide services and build product for our clients. On the other hand it has forced us to have a very clear view of who we are.

Contracts in the supply chain

One of the most difficult things to manage is the perception our end clients have of us. In many cases, our contracts are handled by the client’s HR, who in some cases just don’t see the distinction between a business providing services and rent-a-dev temps who look like employees without the hassle of holiday pay or pensions. This isn’t helped when our connection with the clients is frequently handled by agencies and other intermediaries, so we don’t get to clear perceptions until after the contracts have been signed.

A standard engagement via an agency or consultancy firm has at least two contracts: there’s the contractor to agency contract (the one us freelancers sign on behalf of our companies), and the agency to client contract - frequently called the upper and lower contracts (but I can never remember which is which). A longer supply chain will have more contracts.

I’m a firm believer in having good contracts and sticking to them. Over the years I’ve required many changes to contracts that did not reflect the way my company provides services. I favour the terms generally seen in ‘outside IR35’ standard contracts not because they’re a magic tax get-out-of-jail-free card (they aren’t), but because the same terms that describe an outside IR35 engagement also describes how an independent company like mine operates. That’s why the contracts are often described as outside IR35. It’s not all about the tax.

The difficulty is when the nice clean contract I have signed, which clearly defines how my company is going to provide services, bears little resemblance to the other contracts in the supply chain. These may or may not describe me personally being indistinguishable from an employee on really bad employment terms. I rarely get to see those other contracts so have no idea how my company’s services are being presented to the client. Worst case, the client thinks they have another effective employee on board and it’s an uphill struggle educating them after the contracts have all been signed.

The issue here is that the contracts in the supply chain don’t match up, and us freelancers at one end of that chain rarely get to find that out until there’s a tax investigation and the client during interview says “of course they’re a temp - that’s what the agency/consultancy/whatever sold us”. There are ways to reduce the risk of this, but they involve a hell of a lot of extra legwork on our part and/or the goodwill of our clients.

Off payroll legislation changes

So, as of April 2021 that changed. The legislation and its implementation caused a load of other problems, granted, but it does have one big advantage. For the clients to which Ch 10 now applies (medium to large companies with offices in the UK), they are required by law to provide an IR35 status determination statement (SDS) up front. In my possibly naive view, that means they are required to confirm in writing what their perception of my company is. If a potential client insists on excessive direction and control, and they give a status determination that makes me personally look like an employee of theirs, then the contract just doesn’t happen. There’s less worry about both tax status and whether incorrect perception of what my company does is going to prevent me from doing my job effectively.

Unfortunately it’s tax law, and anything involving money has to get complicated. The difference between Ch 8 and Ch 10 is poorly understood by many agencies and clients. I’ve had to educate entities that insist they’re the client for the SDS when they’re clearly not, those who claim a contract is automatically outside IR35 as the client is overseas, clients making blanket determinations for all contractors as they don’t want the risk, etc.

The SDS can just be a “You’re inside/outside” with little supporting information, which isn’t great. The current commonly used SDS source is the CEST tool, which has its own host of problems (see IPSE or Dave Chaplin for the details there) - unfortunately I think it’s currently the best we’ve got. It may not have the same weighting on business factors as are represented in case law (long story), but it does ask some questions which clearly show the client’s perception of the contractor, and it is said to be honoured by HMRC assuming the client was accurate and the contractor’s working practices match the answers.

Summary

In short, for a freelancer, the requirement to get an SDS up front gives us an opportunity to make sure our contracts show us to be the real businesses we are (at least at both ends of the supply chain). It also sets expectations with our clients as they’re the ones that have to provide the SDS. If a client really does want an effective employee, it’s obvious prior to contract signing. At that point I would either persuade them (via the supply chain) to engage my company as a proper business to provide services, or failing that just walk away. Clean and simple.

Shared at https://www.linkedin.com/pulse/off-payroll-legislation-private-sector-good-points-donal-stewart

Self-isolation - just do it

Fri, 27 Mar 2020 00:00:00 +0000

Debated posting this… I don’t want to just jump on the current topic of interest, but I’m seeing so many idiots around that I wanted to say something.

tl:dr (and in case anyone misunderstands the thought experiment below) - Self-isolate. Doing anything else is being a selfish idiot.

Oh and IANAE (I am not an epidemiologist) so don’t take this as expert advice.

We are in unusual times. The current pandemic at the time of writing has caused a minimum of 24000 deaths worldwide. Governments around the world are mostly getting past the denial phase and entering lockdown mode. We (in the UK anyway) are being told to stay inside, don’t meet up with others, limit exposure as much as possible. Why are the draconian rules (now law) required? Because as a population we are basically morons - we instinctively live in denial ‘I’ll be fine’ or ‘I’m not likely to catch it’ or ‘Yeah, but I’m bored and want to go see my mates’…

Another part is how some of us react to fear and uncertainty - we don’t know when we’re going to get infected, and we don’t know how serious the results of the infection are going to be. Also, we don’t know who we could be infecting while we’re wandering around pre-symptomatic. Some of us (myself included) instinctively want to face down that fear immediately ‘**** it, let’s get it over with. Let’s get sick and be done with it’. Let me go through the logic of why that’s a really bad idea.

Best case - perfect quarantine, mild symptoms

Lets say hypothetically it was possible to get infected with absolute certainty at a time of your choosing. You prep your home for 7-14 days quarantine (depending on whether you live alone or not). You’ve got supplies to last, so you go out and get coughed on. Best case, everyone in your household has a miserable week, and you emerge afterwards immune - you’re invincible. You can’t infect others with your coughs and sneezes… you’re safe. Except…

Even if you’ve had it once, there is no absolute guarantee you’re immune from getting it again. Your immune system has been activated to this particular virus so reinfection is highly unlikely in the short term, but there has been at least one report in Wuhan of someone getting reinfected.
Even if you are now immune, you can still spread the virus. Even if your body isn’t manufacturing it, you can still spread it around from and to others by touch.

Imperfect quarantine, mild symptoms

What if you make a mistake in your quarantine? Someone nearby gets infected because of you. Say they don’t get so lucky with the symptoms, or the people they subsequently infect, and so on. I have elderly neighbours that I intentionally haven’t seen for weeks because I don’t want to risk killing them.

Serious symptoms

There’s no guarantee it’ll just be a flu. Current stats puts the mortality risk for my age group at 0.5-1%, and even healthy, young folk have reportedly died from it. Am I willing to intentionally find out? If there were no knock on effects, actually maybe. However, it has substantial knock on effects on others.

Even if you are fine, is everyone in your household likely to be so lucky? Your partner, your kids - do any of them have any health issues that would increase their risk of complications? That’s not a risk you can take on their behalf.
Even if it was just me, I risk being hospitalised. Even if I survived, I would have been using up resources and medic time that could have been used to save someone else’s life. I don’t want that on my conscience. Even testing in the case of mild symptoms is using up time and resources we don’t currently have to spare.

In summary, the current lockdown we are experiencing is necessary and rational. You should be doing all you can to abide by it. To do anything else is to be a self-centred moron. Unfortunately, there are still far too many of those around. The fact that it had to become enforceable law (and recent news reports) demonstrates just how stupid some of the people living here can be.

Shared at https://www.linkedin.com/pulse/self-isolation-just-do-donal-stewart

Stress - dealing with it

Mon, 16 Mar 2020 00:00:00 +0000

No, I’m not turning into a life coach, honest :)

Just a discussion about stress, at work and in life in general. I reckon I have a pretty stress-free life, accounting for my youthful good looks (stop laughing at the back there) ;)

Much of that is due to my consciously choosing stress avoidance and mitigation work and life patterns. Contracting has the potential for being stressful, unless you actively recognise the causes of the stress (uncertainty, finances, etc.) and mitigate them. I’m generally relaxed about those parts of my job - maybe too relaxed, but it’s always worked out so far :) I’ve never been concerned about finding another contract, regardless of all the scare stories about the sector dying (which I’ve heard on and off for decades), or the government push to kill IT contracting as an industry (yup, IR35). In contract I have fun creating business value for my clients, out of contract I have fun with all the projects of my own that interest me. It’s a good life.

Good and bad stress - good stress is another term for excitement, bad stress is another term for imminent cardiovascular event. What’s the difference? I’d say the primary difference is that the former is caused by events under your control, and the latter is the emotions generated by things happening outside of your control. Bad stress is the frustration between what you want to happen, and what is actually happening and you think you have no means to influence. It’s generated when you feel helpless in a situation with no apparent choices.

I used to get stressed (the bad kind) early in my career, usually when I was in a situation where I didn’t feel confident or experienced enough to handle. I felt it when the projects I worked on hit blocks I couldn’t immediately mitigate.

Nowadays, I rarely get truly stressed. I do get irritated, granted, but am rarely in a situation where I feel out of choices. If I’m in a toxic environment, fundamentally it’s because I chose to be there. Worst case I can choose to not be there - problem solved.

When you realise that the stressful situation you’re in exists because you’re choosing to remain and attempt to deal with it, your perspective immediately changes - it becomes less frustration and more challenge. In the cases where it’s just a clusterf*ck with no apparent remedy, where everyone else seems to be dead set on not fixing the problem for their own reasons, you can just walk away.

Walking away has been an excellent stress relief over the years. If you’re in a bad situation that can’t be resolved through logic and reason, choose to not be part of it any longer. If the other parties to the situation don’t want your help in resolving it, leave them to it, let them implode. You tried, it’s no longer your concern.

I’ve had one or two contracts like that - the environment was just toxic, the people I had to work with were incompetent and/or desperately defending their little feifdoms rather than doing their jobs. They were not fun times, but vastly improved by the realisation that logic or common sense were not going to prevail… time to leave.

I’ve had colleagues in similar situations with different coping strategies, usually keeping their head down, putting up with the crap and continuing billing. If their project failed it wasn’t their problem, they still got paid. I like winning too much to willingly sit on a deathmarch project, so I leave when it becomes apparent I’m in a fight I can’t win. Fortunately that has happened only a couple of times in my career.

When you realise that situations aren’t as lacking in choices as it first appears, stress reduces. A miserable situation can turn into a merely challenging one.

I’ve had a few projects that were high in the other (good) kind of stress. There seems to be a correlation where good stress situations are technical in cause - a project to deliver by a certain date, and bad stress situations seem to be caused by people or process. Projects under tight time pressure are fun. Makes the work days exciting, not miserable.

I used to enter programming competitions with friends earlier in my career - they had much the same feeling. Needing to complete a set of tasks under very tight time pressure. Competing against very smart people worldwide. It was exciting, and had the feeling I try to keep in high pressure contracts. When you’re thinking and typing as fast as you possibly can and you’re laughing, you’ve got the good kind of stress. When you’re swearing and angry instead of laughing, you’re in the bad stress situation.

Steps to mitigate the bad stress:

Identify the cause - it’s not necessarily the simple fact of the workload or the time pressure. Identify the place where you think there is no control or choice - that is most likely the cause of the stress.
Avoid the avoidable causes - if there is a single person or group of people causing the stress, find ways around them. Escalate if necessary - make getting past them your team lead or manager’s problem (that’s why they get paid the big bucks ;) ).
Structure and schedule your workload better - if the primary cause is time - imminent deadlines, triage the work you have to do. Be realistic about work you are capable of doing well (individually or as a team) in the given timeframe. If the essential work can fit into that amount of capacity, focus on that and push the rest to one side.

If you can’t fit the essential work into the capacity, you’re doomed - no point continuing stressing about it. You tried - there’s no point getting a heart attack over the physically impossible. When you make that situation clear to the stakeholders, frequently some of that set of essentials becomes not quite as essential as previously thought. Better to have that conversation early, before the team break themselves.

If the stress is coming from no particular deadline but a steady overload of work, manage it better (yes, not hugely helpful). Learn to say no to stakeholders. Make your planning and capacity more transparent to stakeholders - you have capacity X, you are not going to attempt to do more than X amount of stories per timeframe, regardless of how much they jump up and down.

Once you have rules of engagement with them as to how they can choose the order but not the quantity of work to be done, or when it is acceptable to bump work up the priority queue (to replace existing work, not add to it), things become less stressed. You can then focus on doing the work rather than panicking about all the work you’re not doing. In the long run you’re actually giving them a better customer experience by having a stable, predictable output.
Learn when to cut your losses - in toxic situations, learn when the best response is just walking away. Regardless of how much your office life rules your day and possibly stresses you, in the end it’s just a job. Unless you have signed a really bad contract, you’re not imprisoned there, and there are generally other options available.

It’s easier for contractors, as the mindset of leaving and going somewhere else is our normal business practice - one contract ends, we go get another. I guess leaving is a bit scarier for permanent staff, but the fact is in our industry moving between companies is normal practice. If you’re in a bad, stressful situation in your current location, and you can’t see another way to fix it, polishing up that CV is probably better for your health than staying and being miserable.

It’s down to recognising that there are choices, you are rarely as out of control as you think.

Shared at https://www.linkedin.com/pulse/stress-dealing-donal-stewart

Gaining income

Thu, 12 Mar 2020 00:00:00 +0000

I’m a contractor. Like many or most of my colleagues in the contract sector I believe I have an independent streak - it’s important to me to be in charge of my own fate/career/destiny. If I mess up it’s on me.

One aspect of this is financial responsibility. I need to know exactly how my finances and income are constituted. That’s not to say I need my company to have a stable, dependable income. Any contractor in that position is really in the wrong line of work. Rather I will know exactly how much my company needs to bill in a given timeframe, how much of that eventually gets paid to me. I have detailed projections of my company’s and my financial health over various timeframes and scenarios.

The alternative, as a permanent employee, is to have your employer take care of all that for you. I have issue with those who just depend on their employer to do all their thinking for them - letting them know how much they pay in tax at the end of the year, taking the default pension and health contributions that the employer deems they should have, and progressing up pay scales (or not) as HR decide.

This dependence on others to do your thinking for you and trusting they will have your best interests at heart is dangerous. First of all, they can make mistakes - it’s easily done. Second, an employer rarely has your best interests at heart - they are there to make money. They are rarely going to voluntarily part with more of that money than they have to. If you want a bigger share of that income, you have to fight for it.

The same is true of contracting. A client isn’t going to pay more than they absolutely have to. An agent is going to want as large a cut as they can get away with (or as large an aggregate income as they can manage from a single client). As a contractor you have to be aware of that and negotiate accordingly. Be aware of the value you bring to the client. Know how much that is worth. Be prepared to sell that value at the rate you want, or be prepared to walk away from the negotiations if the client isn’t willing to pay the value of your services.

Market forces are always at work. Know what the going rate for your skills are in your location and sector. On a long running contract, be aware of the value of your accumulated domain knowledge. While the industry as a whole tries to make developers (for example) entirely interchangeable and replaceable (see the CMMI), there is always domain knowledge that can be leveraged. If you’re experienced in a particular system, company or environment, know that that has value.

Be aware of the political and financial landscape around you. As I’m writing this, the spectre of the looming IR35 private sector changes are making the large financial clients panic, and large consultancies are playing them excellently on it. It makes working as a smaller independent contractor in that sector challenging right now. How you handle it is up to you - personally I’m not stressed about it as I think it will settle down soon enough.

Enough of the digression. The main point was in this capitalist economy it’s not enough to just plod along, putting in the hours and being happy with what you are given, contractor or permanent. Your employer or client rarely has your best interests at heart. At best they care just enough to ensure they don’t lose a needed resource. Sticking to that strategy will guarantee you approach retirement with at best average salary, even assuming that employer doesn’t go bust or you get made redundant (if you’re a permanent employee). If you want better than that (in my opinion) you have to go out and get it, fight for it if necessary.

Shared at https://www.linkedin.com/pulse/gaining-income-donal-stewart

Honesty at work

Mon, 09 Mar 2020 00:00:00 +0000

Perhaps another contentious thing to be writing about - I’ll get them all out of the way at once :)

I’m an engineer - I believe a essential component of that is being scrupulously honest. It is a matter of honour for me that I will not lie or dissemble.

Unfortunately in my experience that is not a common trait of all people I have encountered professionally (or elsewhere). You all know what I mean, there are always people who can’t be entirely trusted, bullsh*t artists who talk a good game but can’t back it up, people you just can’t rely on to do what they promised to do.

It’s irritating to have to deal with this - there’s just a dissonance in the logic of what they do. Why lie and damage your reputation? People will notice, people will remember.

For the bullsh*t artists - we’ve all met them. People who will speak knowledgeably about stuff they have no clue about. Who will affirm something is the case without any evidence whatsoever, who will guess rather than simply say ‘I don’t know’. External consultants brought into companies tend to have a (in the most part undeserved) reputation for this.

I’ve had developer colleagues who will claim they have completed a particular piece of work or fixed a piece of code when they just didn’t bother. We have source control, people! How stupid do you have to be to think this isn’t noticeable?

I have been burned too many times in the past dealing with agents to be comfortable taking anything they say at face value. Early in my career as a contractor, I used to trust what I was told. After travelling to another country to start a brand new contract that turned out to not exist, I learned my lesson. Nowadays, I am a lot more cautious.

So what is it about our profession or sector that allows dishonesty and bullsh*t to exist? Are we a special case, or is it like that everywhere?

I have a simple set of principles I apply everywhere:

I will tell the truth if asked a question. If people don’t like the answer, then they shouldn’t have asked. If the item in question is none of their business, I will not be shy in saying so.
I will not dissemble if I don’t know something. I actually make a point of making my lack of knowledge about something known. In part to encourage others that admitting they are not all-knowing is ok, and in part to find out if the matter in question is generally not known and needs to be investigated by someone on the team.
I will do exactly what I agreed to do. If it turns out that something is preventing me from achieving my agreed task or goal, I will let the people I agreed with know rather than hoping the problem quietly goes away.

A related issue is negotiating - for example when settling the terms of a contract. I’ve studied negotiating strategy, and had no end of discussions about it with my colleagues over the years. I have a simple set of negotiating principles, and I don’t think my future negotiating position is weakened by sharing them (and may in fact save me some time):

There are usually two categories of things in a negotiation - the essentials and the nice-to-haves. I might consider dropping the nice-to-haves if necessary - I rarely have those in a particular negotiation, and they’re usually trivial, procedural stuff. The essentials however are (ironically) non-negotiable.
I don’t bluff - not ever. If I take a position on the essential points, I will not back down. To do so is just stupid - it weakens your position and shows how malleable your principles are.
I am always prepared to walk away from a negotiation rather than sacrifice an essential, be it rate, legal terms, whatever. I have decided what are reasonable and fair terms. If a client (or more usually an agency) aren’t willing to meet them, then there’s no point trying to haggle. It’s amusing to see how many try though. I’ve had quite a few contracts where the other party has effectively declined my offer, only to come back with an acceptance out of the blue a few days later after it becomes obvious I’m not going to cave.

A frequent response to my requiring a change in contract terms, etc. is along the lines of ‘It’s our standard contract, no-one else has ever objected’. Rubbish. It’s fairly certain other contractors have objected on very similar grounds - we run our own companies, we know the legal implications of the clauses in contention. If the agency or client in question refuses to make the modifications (there are some known for it), it’s no big deal - there are plenty of others out there.

Shared at https://www.linkedin.com/pulse/honesty-work-donal-stewart

Having a blameless culture... or not

Thu, 05 Mar 2020 00:00:00 +0000

A contentious topic follows - I reckon there may be disagreement with my opinions - feel free to respond :)

I’m currently reading Site Reliability Engineering, describing the DevOps strategies at Google. It’s an interesting book - one item they discuss is their blameless culture in responding to failures. I’ve never worked at Google, so I don’t know how much their reputation for having teams of A-list engineers is true and how much is good PR, but the assumptions that make those practices effective in Google development sites aren’t true in the general case (at least in my experience). Specifically, how much one can rely on the conscientiousness and goodwill of all engineers.

One of the reasons contractors get brought into companies is to provide short term skills boosts for projects in trouble. We get brought in to put out fires, or resource a project that is already in trouble. Sometimes we get to see the companies where everything isn’t all working well. If companies had everything as right as Google seems to have at software engineering, then there would be no need for contractors. The companies would have exactly the right number of highly skilled, highly motivated, competent and conscientious engineers. Wouldn’t that be nice?

The reality isn’t as shiny. Most companies, while on average have teams of pretty good engineers, also have to deal with engineers at the bottom end of the bell curve. These encompass a range of issues - the work-to-rule jobsworths who do the minimum required to get paid, the well-intentioned but careless engineers who don’t check their work and make mistakes, up to the people who knowingly do sub-standard work because they quite frankly couldn’t give a #### about their job, team, product or company. Most of us have seen or had to deal with each of these at one point. If you honestly haven’t noticed anyone like that during your career, I’ve got some bad news for you… ;)

Having a blameless culture is all fine when errors are honest mistakes that slipped though despite the best efforts of everyone involved. I’ve made some howlers in my time, so I am in no way claiming infallibility. In those cases everyone is rightly focused beyond who screwed up, to how it was possible to screw up, and how to prevent it happening (in that way) again.

However when the mistakes occur because someone (or more realistically a whole chain of someones) didn’t do their job right, then the correct response in my opinion is ensure those responsible get their collective asses handed to them (depending on the severity of the failure, and how often those particular people are making mistakes), while simultaneously doing the rest of the postmortem to reduce the risk of recurrence.

The difficulty here is ensuring that the blame is correctly allocated. Indeed that is often so difficult that it motivates the blameless culture in the first place - it’s better to let a mistake due to carelessness (at best) risk recurrence, and allow an environment where the careless, incompetent, or outright lazy can thrive, rather than throw blame at the wrong people.

The approach described in the book I’m reading stresses focussing on events rather than people during postmortems. In a well functioning company or development environment that makes perfect sense. What about the cases where the systemic causes of failures are the people? Not that there weren’t adequate checks and balances at the system level to allow careful teams (in the sense that they care about the product) to detect issues early, but that avoiding those issues just wasn’t a priority for the people involved.

I’ve had to deal with some utter cowboys in the extended teams I work with - people who knew they were screwing up, but also knew they could happily get away with it. It’s not pleasant. In some projects it required us to construct safety nets within and around the team, to catch bugs and mistakes before they could do significant damage. To monitor everything the bad actors touched, and build walls of defensive tests to protect our work from dodgy external (to the team) subsystems. Having to check the work of teams our team interacted with because we just couldn’t trust them to do the job correctly.

It was a resource drain, and a cause of considerable ill-feeling within the team and larger enterprise. The better solution would be to remove the under-performing or rogue elements from the project, but I’ve rarely seen that happen. I have a lot of respect for the team leads and managers that did actually bite the bullet and restructure teams to remove the problem elements.

Paradoxically, one of the things I like about XP in practice is that it doesn’t cope well with bad actors within a team. Yes, pairing and the other practices mitigate the issue in the short term, and potentially motivate the under-performers to up their game - team spirit is strong with XP. However if there isn’t a turnaround then it is common practice to remove such developers from the team. There’s little scope for someone to coast within an XP team.

At the other end of the scale, having a culture where people live in fear of getting blamed for a screw-up (rightly or wrongly) is also unhealthy. I’ve been in teams and environments where that was the working practice. It doesn’t work out well in the long term, the culture is unpleasant, and the more mobile, skilled developers leave to go somewhere nicer.

There has to be a ‘happy’ middle ground, where conscientious engineers can happily get their work done, while also maintaining enough accountability that bad actors cannot thrive in the company.

I’d be interested to know how others deal with these issues. It’s something that has recurred repeatedly over the years, particularly in the larger organisations I’ve worked with. Smaller companies have less scope for these issues, there’s only so much dead-weight a small company can support before going bust. Also, bad actors can’t hide in the system as easily in a small team or company.

Shared at https://www.linkedin.com/pulse/having-blameless-culture-donal-stewart

Information privacy and control

Mon, 02 Mar 2020 00:00:00 +0000

Expanding on a comment in my previous article, I thought I’d talk about how I use web browsers and my general attitude to information security online. Short version - in my opinion it is nuts not to assume that websites and mobile apps are hostile and out to take you for everything you have.

Any tech and privacy savvy developer can tell you just how invasive and pervasive online monitoring, tracking and marketing is. There are routinely stories online about both the invasiveness of online tracking in our everyday life, and the routine breaches in security of these companies which promise to keep your data safe. Such stories appear so often in the daily Slashdot or Register feed that most people just don’t pay attention anymore.

There is only one way to have your personal data safe online - don’t have it online in the first place. I will routinely limit the amount of personal information I give out to the absolute minimum necessary to perform a given task. I routinely use burner emails when dealing with companies in day-to-day life, and it is really interesting the emails I started getting on those supposedly private email addresses. GDPR helped, but how long is that likely to last here?

I will not give out my mobile number to companies. End result, I don’t get marketing spam or cold calls on my mobile… (except from the phone provider, nothing’s perfect). My mobile is running a trimmed down version of Android, and only runs a few apps for which I can see the source code, and in some cases apps for which I have trimmed down the permissions required and rebuilt myself. Does a PDF reader or a text editor really need internet access or to read your contacts list? I don’t install third party apps for accessing bank accounts, transport and all the other connected services used by most people in daily life.

For web browsing there are other preventative options available. I use Firefox routinely as I have the (hopefully well founded) belief that they are less likely to pull marketing bull on their users than the more commercial offerings available. I always run in maximum privacy mode, and clear down cookies and cached data automatically after every session. I have ad and JavaScript blockers installed. All in an effort to stop being used as yet another marketing and profiling data point.

There are other benefits from my paranoia (as my friends call it). For the ad blocking - if you haven’t tried it, you’d be amazed at how much cleaner and faster your web browsing experience is when each page needs to load only a fraction of the full ad-enabled page. There are no distracting banner ads, no webbugs to random marketing and tracking IP addresses. The webpage contains just (or at least mostly) the information you went to the page to find in the first place.

For JavaScript, I believe I have the right to choose what a website can run on my machine. All those third-party scripts for marketing and tracking, I didn’t consent to. Yes it is a pain to whitelist scripts and sites, and in some cases sites require a bewildering list of JavaScript from third-party hosts just to run. Those I would consider dodgy as hell anyway and am inclined to just stop using that site than try to figure out which of the scripts are actually necessary for what I want to do. That’s not even counting the openly hostile malware and miner code that exists online.

One tip for eCommerce developers in dealing with ‘difficult’ consumers like me - have all the necessary eCommerce processing JavaScript self-check at the start of transaction processing. There are a few sites that will assume just because one script is available they all are, regardless of the script host. I’ve had a few transactions fail because the card processing scripts from yet another third-party host break down. If I can’t easily figure out which hosts to whitelist, I’ll just go to a competitor. Your loss.

Yes, I put quite a bit of effort into this, and may seem like a basket case to the more connected, app friendly people here. For a bit more privacy, less spam and a cleaner browsing experience, I can live with that.

Shared at https://www.linkedin.com/pulse/information-privacy-control-donal-stewart

User experience and when it goes wrong

Thu, 27 Feb 2020 00:00:00 +0000

We’re at the stage now where customer experience can be greatly enhanced by IT. Well to be accurate, we’ve been on a continually improving trend for decades. These days we expect instant replies, live chats, automated self-service systems for pretty much every common interaction with businesses. The interactions are usually mapped out, automated and commoditised.

For example, how many of those customer interaction live chat systems are actually bots behind the scenes, at least for the early stages of data gathering from the customer, bringing a real person online only when necessary? Once the conversation pathways have been mapped out, there’s no need to have a person sit at the other end and paste out the same questions again and again. This is an assumption on my part, I haven’t actually seen the internals of call centre handling and live chat infrastructure - it just seems the sensible and economic thing to do with the technology available these days.

So it is that much more irritating when we as end users or customers hit a bug with these systems. It grates that much more as we are just used to so much better. Flawless user experience is that much more important as our expectations have increased over the years.

Case in point - I recently tried to cancel my gym membership (reasons aren’t important - it’s a different, non-IT rant :) ). I was already in a less than salutary mood with the company in question, so had less tolerance than usual for user experience hiccups. I sent the email requesting membership cancellation to the email address supplied in the membership FAQ of the gym. No problem there. I receive a ‘your email is important to us, please wait’ auto-response. OK so far, but here’s where it goes wrong.

First, there was no follow up for a full day. I have no knowledge of the internals of the business and how they set up their customer processing, but given the company is based in the UK, I would have expected the enquiry processing to be operating in the UK timezone (regardless of where it was actually based - that part is irrelevant). No reply for a full day, before I sent a follow up email. Either they are processing a lot of cancellations (the autoreply email mentioned cancellation early in the text, so that might even be plausible), or their customer processing workflow is under-resourced or very inefficient. Either way, not good.

The autoreply email also recommended I log into my online membership account and cancel my membership there. It even provided helpful instructions on where to navigate to find the ‘Cancel’ button. Excellent, I think. I had actually looked for a similar option prior to sending the email, as again I would have expected that interaction to have been added to the online account user experience long ago - it would just have made sense. I couldn’t find it though - maybe I missed something.

I follow the navigation instructions to the memberships page to get to the ‘Cancel’ button. Nope - it’s not there. Ah - must be requiring JavaScript from a site I haven’t white-listed (I block JavaScript by default - to not do so is insane in my opinion). Nope, all JavaScript is enabled on that page - there isn’t even any from the tracking sites I blacklist by default. The page should be fully functional.

Next step for an IT geek, inspect the code backing the page in the browser. It’s dynamically rendered, but where I would have expected the ‘Cancel’ button to be there is an empty table cell. Well, not quite empty - it has a bunch of commented out code which strongly implied that there should have been membership actions available there.

End result, I have a nonfunctional membership page, and an email which gives incorrect instructions. Both of these are a user experience fail, and result in an even more irritated customer than before the interaction.

How to avoid this sort of problem? There are a couple of simple steps relevant to this particular instance:

For every user story, in this case online membership cancellation, have automated regression tests which run against all production deployment candidates. If someone pushes code which breaks that option, it will be caught. If that particular option could possibly be influenced by membership categories or account data, then cover those cases in the test. Simple!
For boilerplate text, in this case the instructions in the autoreply email, but more generally any text or user documentation describing the online system, ensure it matches the changes to that system. Every functional change to the system should also have a review of the surrounding documentation to see if updates need to be made. The dissonance between description and fact grates… a lot.

These are IT and product actions that directly effect customers of a business, even if the business itself isn’t an IT entity. They’re simple steps, but ensure that your customers remain at worst neutral about their user experience, rather than irritated enough to want to talk about it with their friends, or maybe write about it. That can’t be good for marketing and brand reputation.

In the end I bypassed the central membership processing, and emailed a human instead - it got sorted in ten minutes. Happy (or at least acceptable) outcome for me, but with lasting memories of poor user experience with that company. That’s a fail for their IT department (or external suppliers).

Shared at https://www.linkedin.com/pulse/user-experience-when-goes-wrong-donal-stewart

Having fun at work

Mon, 24 Feb 2020 00:00:00 +0000

I love my job. I’ve mentioned this before, but it is just so much fun. A recent contract was the longest I’ve ever spent on a single project. The main reason for that was that it was fun. The work, for the most part was challenging and interesting, and the team was entertaining and mad enough to make every day onsite something I looked forward to. Most mornings I’d be grinning on my way into the office - no small feat before 9am and before the caffeine had hit.

In general, I reckon it’s important for a team to enjoy what they do. It is possible to work and not enjoy it, there are many folk who are in it just to fill the hours and get paid. It’s a shame (in my view) but it’s a valid career choice. It’s a path I have actively avoided though. Contracting gives me the flexibility to avoid being locked into that kind of situation. When a contract ceases to be fun, I just choose to not renew or extend, and go somewhere else.

The same option exists for permies though, it’s easy to move around in our profession, and that’s a good thing. It lets devs vote with their feet when dealing with different corporate environments. The dev resource landscape stays fluid and companies have to be realistic about their corporate policies and remuneration.

That’s not to say I just stay in places where I can goof off, and leave when things get hard. I enjoy the challenge of a difficult or high stress project. For me, that is fun. I’ve had a couple of projects which needed to be delivered under extremely tight time pressure. That was exciting - the pressure both focuses the mind, and removes a lot of the bureaucratic rubbish that would ordinarily slow such work down. On those occasions, my teams delivered on time, and we had fun doing so.

To enjoy the work, a team has to be used to winning. Mentioned before but bears repeating. Success breeds success, and a winning team is just happier and enjoy their work more. A team stuck on a deathmarch project are not having fun. At that point, people will leave.

People’s views on what constitutes fun differ, obviously. There’s enough variety there that most projects have something for everyone. For me, it’s having a challenging problem on a codebase large enough to require a bit of thought in managing it. As I’ve said before, I enjoy working with legacy projects. Taking the opportunity to improve and refactor legacy code is satisfying and in my opinion also cost effective. The code is cleaner, clearer and more maintainable. As a matter of course, the test coverage is generally improved, and overall the quality increases. It’s hard to quantify as a simple cost value, but every hour less spent by future devs in trying to figure out unnecessarily complex code as it was before the refactor adds up.

I doubt I would have stayed so long at that client if I wasn’t having so much fun. The work was challenging, and there were enough fun interactions within and outside the team to make it worthwhile. We got the job done, and there was a lot of laughter while we did it. That’s very cool.

Shared at https://www.linkedin.com/pulse/having-fun-work-donal-stewart

Literature searches

Thu, 20 Feb 2020 00:00:00 +0000

One skill required during the last degree was being able to efficiently review available literature on a topic. The process itself is interesting. At the initial stage it’s as you would expect - list all possible keywords or terms that you think are relevant, then run those searches on PubMed, Scopus, Google Scholar.

You need a decent reference manager. I use LaTeX and BibTeX, so tools like KBibTeX and JabRef are very useful. KBibTeX in particular allowing interesting indexing of the growing reference database - by journal, author, etc.

If you’re searching in a specific field (mine was vesicle recycling in synaptic cell biology), then other patterns become apparent. This field has had long running research, but a comparatively small number of labs involved. It’s then interesting to track the publications of both the labs (basically the principal investigator in biomedical research), and the individual authors, as both labs and researchers tend to follow the same topic for years - this kind of science requires significant time and resource investment. If you collate the literature in this way, you can see the stories and theories proposed by individual labs evolve over time. As researchers transfer from one lab to another over the years, you can see the evolution of their theories as knowledge of both the theoretical and practical experimental protocols spread. It’s pretty cool when you see it.

Some of the indexing sites let you track these connections between authors. It makes for interesting graphs. I had a shot at building my own version, with the intention of showing the evolution of theories, linking the collaboration between researchers and labs towards a common goal, and the time evolution of conflicting theories as each lab sought stronger evidence (or disproving evidence - it’s science) for their theories. I dropped the project though as it became time consuming for not a lot of benefit beyond satisfying curiosity.

One difficulty I had to deal with (and previously discussed here) was that different labs may have very different views on the same topic, each supported by their published literature. It means you can’t work from a single source of truth. Each of the conclusions put forward in the published literature has to be weighed on the strength of the published evidence. Conflicts are difficult or impossible to resolve; it’s just something you have to work around in your own research.

I had also attempted to handle these conflicts by grouping the research under a host of experimental parameters - generally there were differences enough in the protocols used to possibly explain the differences in results and the conclusions drawn. The idea was to have a web of evidence from the conclusions drawn, back through the analysis and observations of individual papers, back to the result datasets used to infer these results, finally back to the experimental protocols and environment used to gather each result dataset.

The intention was to have a navigable path from the conclusions put forward by the various labs back to the raw data supporting it, along with confidence measures for each step of the way. I had hoped that such a graph, suitably annotated, would provide a confidence measure of each conclusion, as individual studies either reinforced or weakened each other’s conclusions. It was a fun idea, but rapidly became a massive undertaking and beyond available resources.

Shared at https://www.linkedin.com/pulse/literature-searches-donal-stewart