Best Practices and tools for DevOps Monitoring
While monitoring preceded DevOps, DevOps has further transformed the software development process to such an extent that monitoring has to evolve as well. The overall pace of software development has increased with DevOps and teams are now automating integration and testing, and deploying software in the cloud with quick timelines and continuous delivery.
With DevOps, there’s more to monitor now, from integration, provisioning, to deployment, teams need to use DevOps monitoring strategies to effectively monitor different aspects of the project.
Best Practices for DevOps monitoring
At SouthLights we choose and recommend this Best Practices for DevOps Monitoring:
Step 1: Collect the Data. You can’t manage what you don’t measure! It is important to take inventory of what is being collected today and align with business and executive teams to collaborate on the aim of the services delivered. Analyze the inventory collection of metrics with questions such as: “Why aren’t we collecting this? How does this fit into our goals? Have you ever seen a failure in X? How often should we be measuring this? How long should we keep it for? Is that important?” Teams also should evaluate how they are collecting the information and consider the best architectural approach for collecting their metrics, including whether pull, push or pull collection methods are better. Once goals are understood and the inventory of data is collected, look at what else your organization should consider collecting.
Step 2: Correlate and Triage. Correlation of data is needed to understand it, but data comes in at different frequencies, in different time frames and from different sources. Work to normalize the data to understand it, perform comparisons against the different incoming metrics and establish baselines for basic service availability. Since organizations aim to go beyond basic service level agreements (SLAs) and offer a high-performing solution, constantly ask what the organization is missing from a data perspective and how it relates to business initiatives. Asking that question from the collection, correlation and triage perspective is critical.
Step 3: Identify Trends. Organizations need to examine historical data to identify trends and take action before issues arise and customers complain. Establish alerting thresholds by outlining what a normal day looks like from a monitoring performance and customer perspective, and then identifying examples of what makes an abnormal day. This ties in with managing infrastructure inventory and understanding safety thresholds for each of the components that potentially could impact the service offering. It’s critical to communicate these findings with teams and business-line managers to prevent service delivery problems from happening and optimize offerings based on identified trends.
Step 4: Notify and Act (Automation). In manual mode, a notification is delivered and then the team reacts. But teams are continually pushed to do things faster, and automation can help. To get to that point, organizations must understand where best to add more automation. How do you gather the right telemetry that delivers consistent answers that the machine can operate rules against, and do you leverage an automated process or notify a person? The desire to improve to a faster process requires a shift from manual to automated practices.
Step 5: Predict (What-if Analysis). If you don’t take the time to go through the first four steps in a methodical way, it’s very difficult to reach this final step without always remaining in a manual mode. To balance costs and availability, it’s critical to discuss with the executive team how to predict customer service consumption (revenue) against the amount your services are going to cost the business.
Tools for DevOps monitoring
Here are some of the best DevOps monitoring tools that at SouthLights we recommend and implement:
Zabbix
Zabbix is the ultimate enterprise-level software designed for real-time monitoring of millions of metrics collected from tens of thousands of servers, virtual machines and network devices. Zabbix is Open Source and comes at no cost. This tool is 19 years old along with 300 000+ installation worldwide. it has capability to monitor anything such as:
Network Monitoring
Server Monitoring
Cloud Monitoring
Services Monitoring
KPI/SLA monitoring
Zabbix uses MySQL, PostgreSQL, SQLite, Oracle or IBM DB2 to store data. Its backend is written in C and the web frontend is written in PHP.
Grafana
Grafana is a tool specially designed for the monitoring of services (DevOps) and the monitoring of sensors (IoT), that is, to do analysis of time series fundamentally in real time.
Focusing on one of Grafana's main applications, system monitoring, we highlight one of its strengths, the use as a data source of the main monitoring services of the three most used public clouds:
These public cloud monitoring services allow us to visualize the different metrics collected in a somewhat cumbersome way, something that can be solved using a tool like Grafana, which provides greater flexibility and customization when creating our own visualizations and even you can integrate different clouds in the same dashboard.
In addition, it can be supplemented with relational databases (MySql, PostgreSQL, Microsoft SQL Server), time series databases (InfluxDB, Graphite) or other data sources. All this being able to visualize different data sources in the same dashboard.
Emerging trends such as microservices, containerization, elastic storage, software defined networking and hybrid clouds are pushing the boundaries of DevOps monitoring. The right monitoring plan can identify and resolve problems before they affect critical business processes and enables customers to plan for upgrades before outdated systems begin to cause failures and outages for users. Contact us and start effectively monitoring your resources.
