We have often seen that one solution does not fit all. It is important to monitor database performance efficiency on the different cloud platforms. The way the cloud platform works on Azure is very different from how it operates on Amazon RDS.
This blog discusses how SQLDM for MySQL uses file-based log monitoring for efficient database monitoring.
Single View Monitoring
In MySQL and MariaDB, many different files contain essential monitoring data. When we monitor one parameter, it is important that we also watch its impact on other parameters and how the overall server is behaving. It is often observed that change in one innocent-looking parameter can negatively impact the entire server.
The performance of the server is also dependent on the workload. As the workload changes, various server resources often come under pressure and the behaviour of the server changes.
If the MySQL servers are hosted on-premises, it is much easier to monitor their files, but it becomes challenging when they are hosted on Amazon RDS.
Monitoring Amazon RDS Log Files for MySQL
Database log files in the Amazon RDS DB engine are possible via AWS Management Console. Amazon RDS API or AWS CLI can either download the log files.
Here are various vital commands listed for AWS CLI and RDS API.
AWS CLS
List all the available log files – describe-db-log-files
Download database log file – download-db-log-file-portion
RDS API
List all the available log files – DescribeDBLogFiles
Download database log file – DownloadDBLogFilePortion
REST APIs
One of the most convenient ways to read log files is with the REST APIs’ help. If you know the name of the DB instance (DBInstanceIdentifier) that contains the log file (LogFileName) and the name of the log file, it is easy to download the log file with the help of REST. Here is the sample command for the same:
GET /v13/downloadCompleteLogFile/DBInstanceIdentifier/LogFileName HTTP/1.1
Content-type: application/json
host: rds.region.amazonaws.com
The command above returns logs file data as a stream. All the logs can be configured to be published on the Amazon CloudWatch log. Publishing on the CloudWatch log requires a custom plugin and a few additional commands. Once the data is in the CloudWatch, be further analysed for inefficiency. For ongoing events, a similar methodology of the CloudTrail can be configured, which can help monitor current happenings.
Solution:
SQL Diagnostic Manager for MySQL
While file-based monitoring is essential for maintaining database efficiency, it can get very cumbersome. This entire process gets old pretty fast for DBAs. While learning technical internals may give satisfaction, it often becomes very mechanical for DBAs when they have to manage multiple instances on the Amazon RDS. I prefer to use SQLDM for MySQL to monitor my database on Amazon RDS. This tool showcases general query, slow query, and error logs in a single view with the help of the RDS REST API. It also uses CloudWatch API to monitor important Amazon RDS OS metrics like CPU and RAM.
