#Botlife - Perils to Pearls - IT Operations goes the RPA way
The other day, I was in deep conversation with a Management Guru, who was talking about how to be more creative and go to the next level of excellence.
An employee can only be creative and far more productive, once he/she is freed of activities and thoughts that bog them down. Be it work-culture or be it work in itself. [sic]
Such a simple yet profound thought. It has always been the goal of every organization to improve business processes by either doing process re-engineering or by automating. Both methodologies have their own share of success and failures. Business process re-engineering is a very laborious and arduous activity. It fails to give you that instant competitive edge and is not easy to implement. The most critical pitfall for re-engineering is, it is restructured to satisfy a handful of customers (not applicable to all).
Automations, on the other hand, are more effective and reap results immediately. They work seamlessly when the process is homogeneous in nature and there are available APIs or connectors. With their unavailability, automations are either difficult to create or are not end-to-end and human interactions are required (bringing in error-rates) to perform tasks.
The question arises, how to solve such a problem?
As introduced in the previous post, Nishant from one of the largest service organization was facing a similar issue where he was facing huge work queues where his team works on heterogeneous processes, where there are multiple applications that they work with.
Being a large service organization brings in the issues of multiple clients. You end up performing the same tasks for all the clients with increasing workload. For an IT Operations team, this becomes even more challenging as they have to deal with multiple systems for multiple clients. In the end, the motivation levels come down, the quality of work comes down, the skills remain stagnant, and frustration of operations creep-in.
At this stage, Nishant reached out to us and was pouring out his frustrations at hand. His team was demotivated and his tickets queues were at an all-time high. To make matters worse, Q4 was a few months to go and he needed to scale-up his operations. The coffee was getting re-ordered at a quicker rate at the shop we met.
To get a better understanding of his pain-points, we visited his office the next day, to have a process walkthrough. Our findings:
- The operations team's work can be classified broadly into two activities, handling tickets (work-items) and monitoring.
- Under monitoring, the team was working on Application monitoring, System logs monitoring, database monitoring, and server monitoring. For a single client, it would take 2 hours of every day by 2 resources to complete the activity!
- Tickets were the main contributor for operational load. On a daily basis, they were getting around 500 tickets a day for a single client. Identifying all the types of tickets flowing in was a herculean task, so we worked on the Pareto principle of 80/20 rule. We identified 3 buckets of tickets which accounted to around 80% of the workload.
- These tasks were system provisioning and de-provisioning, server restarts and creation and deletion of user accounts.
Earlier we discussed ways to improve business processes (in this case IT Operations) and understood that re-engineering and basic automations would fail in case of organization-wide impact or heterogeneous environment. In such a situation, RPA is the most dependable solution. This solution works on computer vision and can also work on server level which enables working with multiple applications seamlessly, hence able to execute in a heterogeneous environment. With the help of visual programming, processes can be mapped and created easily which brings down the time-to-market, hence addressing the delays.
To showcase the capability of RPA we first worked on monitoring activities. Monitoring is more of a time-based activity and less driven by events or triggers. This is a pro-active way to ensure infrastructure related issues are identified and attended to before things spiral-out. It is a mission-critical activity and any drop-in-the-ball can potentially cost millions.
This process being time-based made us schedule the activity at our command center, where processes can be timed, queued. Also, being rule-based and run-book based, creating automation flows became very easy.
The process was, at a given time in the day, log into a Windows, Linux, Citrix, Web, ERP or Mainframes server to do system checks. These are essentially a checklist which the operator has to perform, like event and application logs, disk space checks, files, and folder monitoring. Once all the checks are done, the operator logs the information in an excel and send the document via email to superiors for further action to be taken.
This turned out to be immensely successful as we reduced the time taken to perform the monitoring activity by 85%! Now consider this for all the clients in the organization and the amount of time freed-up for a resource.
Next, we looked at support tickets and began with automating user account creations.
Creation of user accounts is one of the biggest activities for the support teams. Usually, the ticket that arrives, contains an excel sheet with all the relevant information. The operator has to perform multiple activities, multiple times to resolve one ticket. Nishant's team was getting around 100 tickets a day for account creations and deletions where they ended up creating or deleting 150-200 accounts a day.
The operator has to log into Active Directory or any other LDAP framework to create the user in the appropriate Organisational Unit (OU). He/she hereon, continue to add permissions to the user account, create a home directory in the nearest file-server, map it to the user account, create an email (exchange) account, and eventually send out the credentials to the reporting manager of the user via email. All this is an extremely time consuming and mundane task.
RPA - a tool which is capable of surface level automation, can seamlessly log into AD or LDAP and create accounts (or via CLI), add groups and permissions, create home directories, email accounts and eventually is capable of sending out an email to the reporting manager with all the credentials. The cycle time for this whole process was brought down by 80% & had great customer and employee delight as all their accounts were active on Day-1.
Server/service restarts were the most satisfactory automation of all. Over a single shift at work, there are numerous tickets that come for the restart of server or service or application. These happen either due to configuration changes in application/service, for any crash, or to release system memory. For organizations which are responsible for the maintenance of server farms and data centers, this is a major activity. All these are trigger controlled and usually, a NOC/SOC team performs restarts once a ticket arrives. Depending on the nature of the server/service hosted from a server, the operator uses a specific tool to perform restarts. The RPA automation workflow was able to log into the required server via ssh or RDP or any other protocol and execute commands to restart the service or server. Once the restart is successful the "bot" can perform ping to confirm, log the required information in the work ticket and close it or reassign it appropriately. This automation workflow released a considerable amount of workload from Nishant's team in a more secure and dependable way.
Server provisioning and de-provisioning was the most challenging yet satisfying process automation of them all. The biggest challenge for the provisioning team is to diligently follow the requirement and procedures to complete the task. More-often-than-not, the customer complains about few missing configurations. This brings the customer experience down and also increases the re-work that needs to be done. Nishant, had a quality team which checked all the provisioning tickets, further increasing his overheads.
With the help of RPA, we could automate the complete process using computer vision, visual programming, and other inbuilt components which seamlessly connect with all the servers and hosts, and execute commands for configurations. Logging of all the tasks performed and uploading of the same in the ticket was also included in the process for audits.
It really does come down to what is the impact of RPA on IT Operations. Is it measurable? Is it time-saving? Is it cost-saving? Will it have other intangible gains?
By the beginning of Q4, Nishant was a relieved manager.
- His biggest concern, operational load, and work ticket backlog were brought down to a very high extent. The 3 primary ticket categories viz-a-viz, user account creation, server/service restarts, and provisioning and de-provisioning of servers were completely handled by bots, which worked 24x7, and ensured the quality of each ticket was 100%.
- The monitoring activity was scheduled from the orchestrator as it was a time-based process, and no manual intervention was required. This completely offloaded the monitoring process to the bot and freed up resources, who could now concentrate only on resolving tickets.
- With the number of tickets to be resolved also coming down with automation, the employees had more time to develop their skills and be more productive and efficient in their daily works. This resulted in high motivational levels in the team.
- With tickets been worked on 24x7, and with a higher quality of work, the end customer was extremely delighted with the outcome, which resulted in more business opportunities.
- Scaling of operations now became very quick, by only adding more bots to the process, where the training (automations) was already in place. With Q4 around, it became a very critical parameter.
Nishant was beaming by the end of it and was really glad he went the RPA way to resolve this existing issues and support his scaling and flexibility issues for future. He could now get more business to his organization as his resources had enough bandwidth and with renewed knowledge, more eager to take up new challenges. It was a win-win situation for him, his team and his business as well.
The positive impact of this change made his SAP head, Sagar, look up and approach us. SAP automations have traditionally been very difficult to execute with the use of screen-scraping techniques. But, with a repurposed computer vision engine, we not only gathered the information accurately, but we also did all of this without enabling scripting, neither at front-end nor at backend! More about SAP automation in the next article.
Are you facing the same issues that Nishant faced? Are you looking to steamroll your operations and increase business for your organization? Get in touch with me for more information.
"One more thing!" We are coming up with something very cognitive in nature, along with being very intelligent. Keep watching this space for more.