wisdom

Some lessons I've gathered along the way:


restarting solves a lot of problems, "give it some time" even more

The blessed "Have you tried turning it off and on?" and its companion "just wait it out".

the hardest tickets are the ones you learn from the most

When you're immature, you get frustrated when a complicated ticket comes your way and you wish for that same ticket you can do with your eyes closed, exactly as the previous 200 times... But instead, you have to deal with some annoying customer/complicated network topology/ancient technology. This statement includes both the technical complexity as well as dealing with difficult people. It's instinctual to try to avoid these situations. The problem with this approach is that you'll learn a lot less doing only what you know. As usual, progress is located outside your comfort zone.

80 percent of the tickets are 20 percent of the customers

This is to be expected in some cases because bigger customers simply have more locations, equipment and links, so there are higher chances of something breaking. But here I'm talking about those annoying customers whose "Internet is always slow" ('cause they can't differentiate between a shit application/website and a problem with the Internet link), whose "settings are always wrong" ('cause they change their network every three weeks), and whose "email is always malfunctioning" ('cause they can't configure their accounts and passwords properly). It's hilarious when they threaten by saying they'll take their business to our competition. Yeah, I bet they can't wait to have you on board. That would actually both benefit us because we don't have to deal with you anymore, and impede the competition because they now have to. Win-win.

it's not done until it's done

People get anchored too easily. In general, refrain from giving time estimates for when it will be fixed, especially in complicated cases.

it's not done even when it's done

Sure, this issue is resolved. But another one will turn up soon enough.

the smaller the customer, the more annoying the problem

Big companies usually have their dedicated network people or even networking departments, which makes them the easiest to work with because they know the procedures, how things work, how things don't work, and most importantly, how things go from not working to working. Medium-sized companies are workable because they have at least one IT person to whom you can transfer the education of the user on how to connect to the correct WiFi, how to change the email settings, or how to forward a call. With small companies, everything stated before falls on you. Or the alternative: explaining to them that you're not their IT and that they should hire one, which they're always delighted to hear.

match the depth of your initial troubleshoot to the depth of the complainant's technical knowledge

Sometimes laypeople complain just to see if something can be improved, but have no concrete evidence of what exactly doesn't work: "the internet is kinda sluggish", "the call quality is mediocre", "I'm sensing the WiFi signal has gotten worse". If you take these complaints too seriously, you end up chasing ghosts and troubleshooting something that works just fine for the level of service they're paying for - which is usually the most basic grade.

you can't troubleshoot it if you can't configure it

KissCo has a useful model of outlining levels of understanding a particular technology in their exam topics - describe, explain, configure, troubleshoot. You can try skipping steps, but you'll end up with a lot of joyful hours of pain - you can ask me how I know... Because I'll answer you: I got my first job in a troubleshooting role, so inevitably I ended up troubleshooting something I couldn't describe/explain, let alone configure. Aaaaah, the pleasure...

you shouldn't configure it if you can't explain it

I first remember hearing about something similar from Russ White: "just because you can configure it doesn't mean you understand it". This one addresses the unavoidable trap of certifications (I myself also fell into): you want to shortcut it (for good and bad reasons) so you glance over the fundamentals and deeply understanding the material and jump to the syntax and configs which are more represented on the exam; because they easier to test and vendor certs are to some extent advertising brochures.

you shouldn't configure it if you can't explain it 2

The other reason is that if you don't understand how it works, how it breaks and what it's designed for; you certainly don't understand how it affects the rest of the network - both working as it should and breaking as it shouldn't, but most certainly will. Sometimes it's about what you don't configure.

the fifth level is design

There's a reason network architect/designer positions are among the most senior roles. Young people (speaking from my friend's experience...) are captivated by sexy job titles like "architect", "red teaming", "cybersecurity"... These are not entry-level roles. "I want to get my ENWLSD so I can design networks." Yeah, sure. The ironic part is that it's regarded as the easiest CCNP Enterprise concentration cert. Operating, breaking, troubleshooting and fixing a lot of networks provide design insights.

you lead the conversation

In general, you'll be the one with more knowledge in this particular domain, so you're in charge of the conversation. People like to be taken care of. They've come to you, you're there to do this professionally, they're the customer. You don't go to your mechanic and tell him how to fix your car. Depending on the customer, on the other side can be an IT/network person with whom you'll be more on an equal level and troubleshooting efforts will be mutual. However, it shouldn't happen that the other side is more competent than you at your job.

you lead the troubleshoot

During the troubleshoot, be wary of the other side trying to help, and as a result, leading you down the wrong trail. This one's especially important if you lack experience.

after the fix, wait 10-ish minutes before checking with the customer

This allows the network to converge and machines (and people) to realize that something has changed. If you call the customer immediately, then you waste time waiting for them to check if they can browse the web, if the POS system is working, have the phones updated their status bar... If you wait a bit, by the time you contact them, they will have already figured out the network is fixed.

disconnect and reconnect please

It's a real hassle to get someone who isn't sure which OS they're using ("the regular one I thinkkk?") to provide you with their IP or MAC address (please don't ask me how I know). If they're connected by wire, then almost always their distance to the router/switch doesn't matter, so you shouldn't care anyway. But you care if they're connected by WiFi and you're responsible for it. Instead of torturing yourself by explaining to them how to get to the CMD prompt and then spelling ipconfig, the more elegant solution is asking them to disconnect and reconnect again, which they're usually capable of. You can then determine where they are using logs, client uptime, or which device disappeared briefly on the connected devices list.

the hardest part of labbing is setting up

Some random error when installing a VM, software image impossible to find, installed version of the program doesn't have the feature you want...

labs can only get you so far

Labs are useful to test solutions in a safe environment (All network engineers have test enviornments. Some even have production.). But some things aren't labbable due to size, complexity, cost... Eventually you have to push to production.

labs can only get you so far 2

Labs are useful for learning. I enjoy labbing and understanding technologies from the ground up. But you learn the most on the job. For one, because some things aren't labbable (at least not at full scale); and secondly, there you'll encounter a mix of diverse technologies from different vendors with various perspectives, and when you add imaginative customers to that mix, you get something that never would have occurred to you on your own.

the network isn't your "baby"

The network is a collection of devices linked together meant to communicate. Anthropomorphizing the network by calling it your "baby" develops an emotional attachment which in turn makes it harder to modify it upon realization that your "baby" is complicated, ugly and deformed. Plus, it's cringe (and not in a good way).

the resource isn't dry, you're just too soft

I occasionally come across comments about high-quality courses, books or resources that say things like "too dry for me" or "I learn better through videos." Even if the idea of learning styles were true (which is debatable), that's a poor attitude to have if your goal is to learn or improve your craft. The aim of these courses isn't to entertain you, but to teach you. So sink your teeth in - you'll come out stronger and smarter.

those fuckin dashboards

Medium-sized companies usually have an IT person or two who isn't a network person but is in charge of the network. The company "invested" in some SDN solution with a shiny dashboard. The complaint goes as follows: "Our dashboard is red and sometimes orange for this location, plz fix". When you ask them what the dashboard measures and how it examines the network status (latency, bandwidth, humidity of the server room, wind velocity), they usually respond: "red, sometimes orange... but all the other locations are green or yellow, plz fix".

sometimes it just fuckin sucks...

In a (strictly) troubleshooting role, you have the privilege of bearing the blame for every bad interaction the customer has had in the past with your company (hey, who doesn't adore their ISP?). Maybe during service activation their questions weren't properly answered, the service isn't working exactly as they imagined, they're being overcharged for subpar service, or they'd already had the pleasure of interacting with the customer service IVR and then waiting on the phone for 10 minutes for an agent who'll listen to them half-assedly for 2 minutes... And eventually they land on you: the Internet doesn't work, the phone doesn't ring, the WiFi is terrible, etc. Rest assured they'll absolutely seize this chance to let you know about everything that pissed them off.

...keep the conversation troubleshoot centered...

"My superiors in Germany/US/Hogwarts will hear about your lousy company!", "I'm 10 times smarter than every single one of you!", "I masturbate every Tuesday evening with your CEO!"... Emotionally incompetent people express their frustrations inappropriately. This partially speaks to their intelligence as well, because it's real smart to insult someone in charge of solving your problem. I straight up just ignore these ramblings and steer the conversation towards the troubleshoot - "how can we get this router working?", "when can you check the LEDs?", "who can the technician contact after arriving at the location?"...

...and you're not their therapist...

You should listen to what they're saying and be empathetic. Here comes BUT. I've seen online some customer service advice that's just plain bullshit that translates to "be an agreeable punching bag, let them vent, listen to everything they have to say...". No, fuck off. I have other customers who'll be grateful for my effort. Criticism and suggestions are always welcome (or at least tolerable when it won't change anything), but lashing out, not gonna happen. I'm not even sure the big company compartmentalization, bureaucracy, inertness (and "revolving door" in this particular department) are even solvable, like by the laws of physics. Soooo, when they start their monologue and you know where they're going with it because you've heard it too many times before, save both of you some time by interrupting and keep the conversation troubleshoot centered: "There certainly have been mistakes made by us, I understand the frustrations, but how can we solve this concrete case?" Occasionally however, they're insufferable. Me, personally, I take pride in being the last person to hang up the phone. I do this every day; you're on my home turf, and you're not gonna win. Or you can hang up the call and let them chill a bit. Sometimes they just need to cool down and call later. Don't worry, they gonna.

...but they want you to be...

They (customer, Sales, technician...) are gonna pester you. Almost always it's counterproductive. Useful phrases: "At the moment I can console you and treat the symptom or I can resolve the root cause by fixing the issue", "the longer we're on the phone, the later the issue will be resolved", "now I could be doing one of two things: fixing your issue or having this conversation with you - your choice". If you're lucky, they're gonna hang up the phone and leave you alone so you can focus on resolving the problem.

...rarely are you the hero...

Even when you solve a problem they're having, the common reaction is a neutral one (which is totally fine in my book), or even better (tacit or not): "Yeah, you'd better fix it; it wasn't supposed to be broken in the first place!"

...but not never

It's nice.