IoT Platform Architecture Design

Overview of Architecture Design

As a bridge connecting physical devices and the digital world, the architecture design of the IoT platform is crucial for achieving efficient, stable, and secure device management. An excellent IoT platform should have the following basic functions to meet the needs of ordinary developers:

Multi-protocol support: The platform must be able to receive and process data from different devices using different communication protocols. This means that whether the device uses MQTT, CoAP, HTTP, or any other protocol, the platform can be compatible and effectively receive data.

Data processing capabilities: The received data needs to be effectively parsed and converted for storage, query, statistics, and analysis. The platform should provide powerful data processing capabilities, including real-time data analysis and visualization, to help developers gain insights from the data.

Flexible alarm and notification mechanisms: When data is abnormal or reaches specific conditions, the platform should be able to issue alarms in a timely manner and notify relevant personnel via email, SMS, or other means. In addition, developers should be able to easily export data for further analysis or reporting as needed.

Fine-grained permission management: To ensure data security and compliance, the IoT platform needs to provide fine-grained permission management functions, including data permission management, user management, role management, and department management. This way, developers can control who can access specific data or perform specific operations.

Comprehensive data security measures: Data security is the top priority of the IoT platform. The platform needs to implement comprehensive data security measures, including data encryption, secure storage, regular backups, and rapid recovery, to ensure the security and integrity of the data.

At the architectural level, the IoT platform can be divided into the following key layers:

• Communication layer: Responsible for handling data transmission between devices and the platform, ensuring the real-time and reliability of data.
• Data parsing layer: Converts the received raw data into structured data, providing a basis for subsequent processing.
• Data storage layer: Provides stable and efficient storage solutions for massive device data.
• User management layer: Manages user information and permissions, ensuring that users can safely access and manage the platform.
• Data permission layer: Controls different users’ access to data, ensuring data security.
• Data analysis layer: Provides data analysis tools to help developers extract valuable information from the data.
• Data storage layer: Used to store collected data and basic system data.
• Notification layer: Responsible for notifying relevant personnel of alarm information, including email, SMS, phone, etc.
• Operation and maintenance layer: Responsible for monitoring, maintaining, and optimizing the platform to ensure the stable operation of the platform.

Through such an architecture design, the IoT platform can not only meet the needs of ordinary developers in device connection, data processing, and user management but also provide strong security guarantees and flexible scalability, supporting developers in building innovative IoT applications.

Communication Layer

In the IoT development platform, the communication layer is the key layer connecting devices and the platform. It is responsible for handling data transmission between devices and the platform, ensuring the real-time and reliability of data. Common IoT development platforms need to support the following communication protocols:

• MQTT: A lightweight publish/subscribe message transport protocol suitable for data transmission in low-bandwidth and unreliable network environments.
• CoAP: A lightweight protocol based on UDP, suitable for resource-constrained IoT devices.
• HTTP: A widely used application layer protocol suitable for scenarios requiring reliable transmission.
• WebSocket: Provides full-duplex communication, suitable for scenarios requiring real-time data interaction.
• Modbus: A communication protocol commonly used in the field of industrial automation, suitable for industrial control systems.
• TCP/IP: Provides reliable data transmission, suitable for scenarios requiring high reliability.

Communication Layer - MQTT Protocol

When integrating the MQTT protocol into IoT projects, choosing the right MQTT Broker is key. Here are some popular MQTT Broker options and key factors to consider when choosing:

• EMQ X: A high-performance, scalable MQTT Broker that supports multiple protocols and plugins, suitable for scenarios requiring high concurrency processing. Official website: https://www.emqx.io/
• HiveMQ: Provides community and enterprise editions, supports MQTT 5, and has complete functions and enterprise-level features such as data persistence and cluster support. Official website: https://www.hivemq.com/
• VerneMQ: A distributed MQTT Broker focused on reliability and scalability, suitable for large-scale IoT deployments. Official website: https://vernemq.com/
• RabbitMQ: Although RabbitMQ itself is not an MQTT Broker, it supports the MQTT protocol through plugins and is a widely used open-source message broker that supports multiple messaging protocols. Official website: https://www.rabbitmq.com/
• Mosquitto: An open-source MQTT Broker that is lightweight and easy to configure, suitable for small to medium-sized IoT projects. Official website: https://mosquitto.org/

MQTT Broker Comparison

Considerations when choosing an MQTT Broker:

1. Performance and Scalability: Ensure the Broker can handle a large number of concurrent connections and message throughput.
2. Security: Check if it supports TLS/SSL encryption and authentication to protect data transmission security.
3. Cluster Support: Evaluate whether the Broker supports distributed deployment for load balancing and high availability.
4. Management Tools: Whether the Broker provides easy-to-use management interfaces and monitoring tools to simplify daily operations.
5. Protocol Support: Confirm the MQTT protocol versions supported by the Broker, including compatibility with the latest version MQTT 5.0.

Recommendation for Enterprise-level IoT Platforms:

For enterprise-level IoT platforms seeking high performance, reliability, and professional technical support, EMQ X or HiveMQ is an ideal choice. These commercial MQTT Brokers provide powerful data processing capabilities and comprehensive security features to meet the needs of enterprises in large-scale deployments.

In the development practice of IoT projects, the MQTT protocol relies on an efficient topic subscription-publishing mechanism to complete data exchange. This mechanism allows devices or applications to subscribe to topics of interest to receive messages published under related topics. Therefore, in the development of software applications, developers must build a fully functional MQTT client management module.

Next, consider a question: Can an application create countless MQTT clients? The answer to this question is, of course, no. Therefore, the implementation of the MQTT client management module can be considered from the following points.

1. The number of MQTT clients in a single application needs to be limited based on certain external factors. Common external factors include:

   1. The memory size of the application
   2. The CPU usage of the application
   3. The network bandwidth usage of the application
   4. The number of threads in the application

2. Unified management of MQTT clients is required, including creation, destruction, monitoring, and exception handling. Since the number of MQTT clients in a single application is limited, it is also necessary to consider multi-application (multi-instance) deployment issues, multi-instance load balancing issues, and multi-instance failover issues.

3. Consider the specific tasks executed in the MQTT client. If the data push interval is very short, allowing the MQTT client to perform some heavy tasks will exacerbate the resource usage of the application. Therefore, it is recommended to directly distribute the data through the message queue in the MQTT client for decoupling.

Next, an MQTT client management solution based on Redis will be introduced. The detailed operation is as follows:

1. Construct two list data structure keys, one named use and the other named no_use. The former represents used, and the latter represents unused. The basic information in these two keys is as follows:
   1. Current MQTT client ID
   2. MQTT Broker IP address
   3. MQTT Broker port
   4. Authentication information (account password or others)
   5. Topics to subscribe to
2. The application sets the maximum number of MQTT clients through command line, configuration files, etc., at startup.
3. External programs interact with all started applications by calling APIs (can be distributed through Nginx). External programs construct request parameters based on MQTT client ID, MQTT Broker IP address, MQTT Broker port, authentication information (account password or others), and topics to subscribe to.
4. Any one of the started applications receives the request and performs the following operations:
   1. Using a load balancing algorithm, here using the simplest least priority algorithm, select an application for processing.
   2. Forward the request to the selected application to complete the creation of the MQTT client.
   3. The selected application needs to establish a mapping table to maintain the relationship between the application and the MQTT client.
   4. Store the successfully created MQTT client in the use key of Redis.

上述操作流程对于MQTT客户端的管理是极简的，但是存在一个问题，那就是当应用程序宕机后，所有与该应用程序关联的MQTT客户端都会被转移到no_use键中，这样会导致MQTT客户端无法正常工作。因此，我们需要实现故障转移机制。以下是一个可行的方案：

1. 每个应用程序都应当周期性的向Redis中写入自己还存活的信息，这是一个具备过期时间的键。
2. 每个应用程序都应当对第一点中提到的键进行过期监听，当发现过期后则认为当前应用程序已经宕机。宕机后执行流程如下：
   1. 通过应用程序和MQTT之间的关系键找到宕机应用的关联MQTT客户端，将其转移到no_use键中。
   2. 通过负载均衡算法，这里使用最简单的最小者优先算法，选择一个应用程序进行处理。
   3. 将宕机应用的关联MQTT客户端转移到选中的应用程序中。
   4. 选中的应用程序需要建立映射表来维护应用程序和MQTT客户端之间的关系。
   5. 将创建成功的MQTT客户端存储在Redis的use键中。

有关于MQTT客户端管理方案的设计告一段落，下面需要关注不同语言的MQTT客户端的实现，具体候选内容可以参考:

1. Java:
   • Eclipse Paho Java Client: https://github.com/eclipse/paho.mqtt.java
   • HiveMQ MQTT Client: https://github.com/hivemq/hivemq-mqtt-client
2. Python:
   • paho-mqtt: https://github.com/eclipse/paho.mqtt.python
   • gmqtt: https://github.com/wialon/gmqtt
3. JavaScript/Node.js:
   • MQTT.js: https://github.com/mqttjs/MQTT.js
   • Paho MQTT JavaScript Client: https://github.com/eclipse/paho.mqtt.javascript
4. C#/.NET:
   • MQTTnet: https://github.com/chkr1011/MQTTnet
   • M2Mqtt: https://github.com/eclipse/paho.mqtt.m2mqtt
5. Go:
   • Paho MQTT Go Client: https://github.com/eclipse/paho.mqtt.golang
   • gmqtt: https://github.com/DrmagicE/gmqtt
6. C/C++:
   • Eclipse Paho C Client: https://github.com/eclipse/paho.mqtt.c
   • mosquitto-cpp: https://github.com/eclipse/mosquitto
7. Ruby:
   • ruby-mqtt: https://github.com/njh/ruby-mqtt
   • paho-mqtt: https://github.com/RubyDevInc/paho.mqtt.ruby
8. PHP:
   • phpMQTT: https://github.com/bluerhinos/phpMQTT
   • Mosquitto-PHP: https://github.com/mgdm/Mosquitto-PHP
9. Rust:
   • rumqtt: https://github.com/bytebeamio/rumqtt
   • paho-mqtt-rust: https://github.com/eclipse/paho.mqtt.rust
10. Swift:
    • CocoaMQTT: https://github.com/emqx/CocoaMQTT
    • MQTT-Client-Framework: https://github.com/novastone-media/MQTT-Client-Framework

Communication Layer - CoAP Protocol

When integrating the CoAP (Constrained Application Protocol) protocol into an IoT project, choosing the right CoAP library implementation is key. Here are some popular CoAP library options:

• Californium: An open-source CoAP implementation developed by the Eclipse Foundation, written in Java, offering rich features and good scalability. Suitable for projects requiring high performance and reliability. Official website: https://www.eclipse.org/californium/
• Libcoap: A lightweight C language CoAP implementation, suitable for resource-constrained embedded devices. It provides the core functions of the CoAP protocol and is easy to integrate into existing projects. Official website: https://libcoap.net/
• Go-CoAP: A CoAP library written in Go, supporting both client and server functions. It provides a simple API, suitable for building high-performance CoAP applications. Official website: https://github.com/plgd-dev/go-coap

CoAP service is a long-running service that is both a request-response model and an observer model. If using the request-response model, it can be designed as follows:

1. Use an open-source CoAP library to implement the CoAP server.
2. Create multiple CoAP servers with a reverse proxy tool (Nginx) to achieve load balancing and provide unified CoAP services externally.
3. The device side sends data through the CoAP client.

If using the observer model, its specific implementation can refer to the MQTT client management scheme.

Communication Layer - HTTP Protocol

Although HTTP (Hypertext Transfer Protocol) was initially designed for web applications, it is also widely used in IoT projects, especially on devices that are not so resource-constrained. Choosing the right HTTP server is crucial for building efficient IoT applications. For web application development, the following frameworks can be used:

1. Django: A high-level web framework based on Python, providing a complete development stack. Its “batteries included” philosophy and powerful ORM make it very suitable for rapid development of complex IoT backend systems. Official website: https://www.djangoproject.com/

2. Flask: A lightweight Python web framework with high flexibility, suitable for building small to medium-sized IoT applications. Its simple API and rich extension ecosystem make the development process more efficient. Official website: https://flask.palletsprojects.com/

3. Express.js: A web application framework based on Node.js, known for its simplicity and flexibility. It is particularly suitable for building fast, lightweight APIs, making it ideal for data processing and device communication in IoT projects. Official website: https://expressjs.com/

4. Spring Boot: A powerful framework based on Java, providing the ability to quickly build standalone, production-grade Spring applications. Its auto-configuration features and rich ecosystem make it an ideal choice for building IoT backend services. Official website: https://spring.io/projects/spring-boot

5. Gin: A web framework written in Go, known for its high performance and lightweight. Gin’s fast speed and low memory footprint make it very suitable for building high-concurrency IoT API services. Official website: https://gin-gonic.com/

Web applications are long-running services that follow a request-response model and can be designed as follows:

1. Use an open-source web application development library to implement web application services.
2. Create multiple web application services with a reverse proxy tool (Nginx) to achieve load balancing and provide unified web services externally.
3. The device side sends data through the HTTP client.

Communication Layer - WebSocket Protocol

WebSocket is a protocol that provides full-duplex communication over a single TCP connection, particularly suitable for IoT applications that require real-time data exchange. Unlike the request-response model of HTTP, WebSocket allows the server to push data to the client actively, which is very useful in IoT scenarios. Here are some popular WebSocket libraries and frameworks:

• Socket.IO: A powerful JavaScript library that supports real-time, bidirectional, and event-based communication. It can automatically downgrade to other transport methods when WebSocket is not available. Official website: https://socket.io/
• ws: A simple, efficient, and pure WebSocket implementation suitable for the Node.js environment. It is the foundation of many other WebSocket libraries. Official website: https://github.com/websockets/ws
• Spring WebSocket: Part of the Spring Framework, providing WebSocket support for Java applications and seamlessly integrating with other Spring components. Official website: https://docs.spring.io/spring-framework/docs/current/reference/html/web.html#websocket

The specific implementation of WebSocket applications can refer to the following process:

1. Use the selected WebSocket library to implement the WebSocket service.
2. Create multiple WebSocket service instances and use a load balancer (such as HAProxy) to achieve load balancing.
3. Implement simple connection management:
   • Perform regular cleanup tasks to remove long-inactive connections.
   • Implement a basic heartbeat mechanism without complex connection pool management.
4. Design the message processing flow, including message parsing, validation, and routing.
5. Implement a security authentication mechanism to ensure that only authorized devices can establish WebSocket connections.

Communication Layer - Modbus Protocol

Modbus is a communication protocol widely used in industrial automation and control systems. Integrating Modbus protocol support into an IoT platform allows the platform to communicate with a large number of industrial devices and sensors. Common SDKs developed based on the Modbus protocol include:

1. libmodbus: A cross-platform Modbus library that supports RTU and TCP. Official website: https://libmodbus.org/
2. pymodbus: A Modbus library implemented in Python. Official website: https://github.com/riptideio/pymodbus
3. jamod: A Modbus library implemented in Java. Official website: http://jamod.sourceforge.net/

In addition to choosing open-source libraries, it is also necessary to consider support for Modbus variants. Common Modbus variants include:

1. Modbus RTU: Suitable for serial communication
2. Modbus TCP: Suitable for Ethernet communication
3. Modbus ASCII: Suitable for certain specific devices

As a software developer (server-side development), it is more inclined to implement the Modbus TCP protocol because the Modbus RTU protocol requires serial communication, which is less common on servers.

Communication Layer - TCP/IP Protocol

TCP/IP is the fundamental protocol of the Internet and also plays an important role in IoT platforms. Although many high-level protocols (such as HTTP, MQTT) are based on TCP/IP, sometimes we also need to use TCP/IP directly for communication. Common TCP/IP development SDKs include:

1. Boost.Asio: A cross-platform C++ library that provides support for the TCP/IP protocol. Official website: https://www.boost.org/doc/libs/release/libs/asio/
2. Netty: An asynchronous event-driven network application framework based on Java, supporting the TCP/IP protocol. Official website: https://netty.io/
3. lwIP: A lightweight open-source TCP/IP protocol stack suitable for embedded systems. Official website: https://savannah.nongnu.org/projects/lwip/
4. libuv: A cross-platform asynchronous I/O library that supports the TCP/IP protocol, commonly used in Node.js. Official website: https://libuv.org/

When using the TCP/IP protocol directly for communication in an IoT platform, the following points need to be noted:

1. Connection management: Implement the establishment, maintenance, and closure of connections to ensure the stability and reliability of the connection.
2. Data transmission: Design efficient data transmission mechanisms to ensure data integrity and timeliness.
3. Security: Implement data encryption and identity authentication to prevent data leakage and unauthorized access.
4. Compatibility: Ensure compatibility with different devices and operating systems, supporting various network environments.
5. TCP/IP, as a low-level protocol, requires high technical skills from developers, requiring strong network programming capabilities.

By using the TCP/IP protocol reasonably, efficient and reliable IoT communication can be achieved to meet the needs of various application scenarios.

Communication Layer - Security Authentication

The previous sections introduced the protocols used in the communication layer. Here, we will focus on security authentication in the communication layer. In the MQTT protocol, the implementation methods of security authentication include:

1. Username and password authentication: Assign a unique username and password to each device, and authenticate during connection establishment. After authentication, the device can transmit data; otherwise, the connection will be disconnected.
2. Certificate authentication: Use SSL/TLS certificates for mutual authentication to ensure the security and integrity of data transmission. Both the device and the server need to hold valid certificates and authenticate each other during connection establishment.

For other protocols, such as CoAP, HTTP, WebSocket, etc., developers need to implement the authentication module themselves. Common authentication modes include:

1. Account and password authentication: Assign an account and password to each device and authenticate after the connection is established. After authentication, data transmission is allowed; otherwise, the connection is disconnected.
2. Token authentication: The device obtains a token through account and password during the first connection and uses the token for authentication in subsequent connections. The token has a validity period and is regularly updated to ensure security.
3. OAuth authentication: Use the OAuth protocol for authentication. The device obtains an access token through the OAuth server and uses the token for data transmission. Suitable for scenarios requiring third-party authentication. This scenario is more often used for web application authentication and is less used in IoT platforms.

By designing and implementing a reasonable security authentication mechanism, the data security and device communication reliability of the IoT platform can be effectively ensured.

Data Parsing Layer

The main function of the data parsing layer is to parse and convert the data transmitted through various communication protocols into a human-readable format or a format that conforms to the system design. In the data parsing process, script languages such as JavaScript, Lua, and Python are usually used. To ensure the efficiency and consistency of the parsing process, the following design considerations are made from a program design perspective:

1. Unified design of return results: Ensure that all parsers return data in a consistent format for subsequent processing and analysis.
2. Abstract design of executors for different script languages: Provide a common interface that supports parsers in multiple script languages, making it easy to extend and maintain.

The following is a unified data return result.

Multi-dimensional data joint warning execution chain:

1. The original data is parsed by the parsing script to obtain the data to be analyzed.
2. Find the joint alarm conditions corresponding to the signal and determine whether the conditions are met.
3. If the conditions are met, a message notification is sent.

Statistical Design

Next, we will design the statistical content. Common statistics are triggered by scheduled tasks, and the execution logic of scheduled tasks is generally as follows:

1. Scheduled task trigger
2. Query the signal values that need to be counted according to the time range
3. Write specific statistical calculation rules
4. Store the statistical results

Extract some core parameters based on the above logic:

1. Scheduled task execution time (CRON expression): The execution time of the scheduled task is the end time of the time range
2. Lead time: The start time of the time range is obtained by subtracting the lead time from the execution time of the scheduled task
3. Calculation script: Write specific statistical calculation rules through script language

In this design, the execution time of the scheduled task is determined by the CRON expression, which is used to set the execution time of the statistical task. The scheduled task is implemented through the expiration message of the message queue, instead of using distributed scheduled task frameworks such as XXL-JOB, in order to pursue lightweight deployment.

Visualization Design

The previous designs have completed the accumulation of data, and now it is necessary to show the relevant data content to the user. You can choose open source components for visualization, and the specific ones that can be used are:

1. Data screen: It is recommended to use the open source ECharts component for data screen display. ECharts is a powerful data visualization library that supports various chart types and interactive functions, suitable for building complex data screens. Official website: https://echarts.apache.org/
2. Data dashboard: It is recommended to use the open source Grafana component for data dashboard display. Grafana is an open source monitoring tool that supports multiple data sources and rich visualization plugins, suitable for real-time data monitoring and analysis. Official website: https://grafana.com/
3. Data report: It is recommended to use the open source JasperReports component for data report display. JasperReports is a powerful report generation tool that supports multiple data sources and complex report design, suitable for generating reports in various formats. Official website: https://community.jaspersoft.com/
4. Custom screen: It is recommended to use the open source DataV component for custom screen display. DataV is a data visualization platform that provides rich visualization components and flexible configuration options, suitable for building personalized data screens. Official website: https://datav.aliyun.com/

Data Storage Layer

This section will analyze the content of the data storage layer.

1. Conventional relational data storage: Choose MySQL or other relational databases. Relational databases are suitable for storing structured data, providing powerful query and transaction processing capabilities, and are suitable for storing conventional data such as user information and device information.
2. Signal point data storage: Choose InfluxDB, or other time series databases. Time series databases are designed to handle time series data, suitable for storing sensor data, log data, etc., and provide efficient write and query performance.
3. Alarm history data storage: Choose MongoDB for storage. MongoDB is a NoSQL database suitable for storing unstructured and semi-structured data, supporting flexible document models, and suitable for storing alarm history records and other data.
4. Statistical data storage: Choose MongoDB for storage. The flexibility and scalability of MongoDB make it suitable for storing various statistical data, supporting complex queries and aggregation operations, and facilitating data analysis and report generation.

When designing the data storage layer, database and table partitioning is an important consideration. Taking InfluxDB as an example, if it is a project oriented to enterprises, the concept of organization in InfluxDB can be utilized. Create an organization for each customer and create different buckets under the organization for data isolation. Usually, buckets can be sharded according to the number of devices to improve data management efficiency.

For MongoDB, since it does not have the concept of organization, developers need to implement a similar organization management mechanism by themselves. It can usually be achieved in the following ways:

• Use different databases to represent different organizations.
• Use different collections in each database to store similar data.

This design method can effectively achieve data isolation, ensure that the data of different customers will not be confused, and facilitate management and expansion.

Notification Layer

This section will analyze the notification layer in detail. The platform’s notifications are usually divided into two categories: in-system notifications (site messages) and out-of-system notifications. The most important components of the notification layer are as follows:

1. Message template, generally using ${} for variable replacement.
2. Message channel, using Feishu, DingTalk, WeChat, SMS, email, etc. for message notification.

Message Template

The message template is the core part of the notification system, which defines the format and content of the notification. By using placeholders (such as ${}), specific content can be dynamically replaced when sending notifications. Common message templates include:

• Alarm notification template: Used to send alarm information when the device is abnormal or the data exceeds the threshold.
• System notification template: Used to send information such as system updates and maintenance.
• User notification template: Used to send information such as user registration and password reset.

Example template:

告警通知：
设备 ${device_name} 在 ${timestamp} 发生了 ${alert_type} 告警，当前值为 ${current_value}。
系统通知：
尊敬的用户，系统将在 ${maintenance_time} 进行维护，期间可能会影响您的使用，请提前做好准备。
用户通知：
您好，您的账号 ${username} 已成功注册。请点击以下链接激活您的账号：${activation_link}

Message Channel

Message channels are the transmission media of the notification system, and different message channels are suitable for different notification scenarios. Common message channels include:

• Feishu: Suitable for instant communication and notifications within the enterprise.
• DingTalk: Suitable for instant communication and notifications within the enterprise.
• WeChat: Suitable for a wide range of user groups, supporting personal and enterprise notifications.
• SMS: Suitable for important emergency notifications to ensure that users can receive them in time.
• Email: Suitable for formal notifications and long text content.

Operation and Maintenance Layer

The operation and maintenance layer is an indispensable part of the IoT platform, mainly responsible for the monitoring, management, and maintenance of the platform. The design of the operation and maintenance layer needs to consider the high availability, scalability, and security of the system. The following are several key components of the operation and maintenance layer:

1. Monitoring system
2. Backup and recovery

Monitoring System

The monitoring system is the core of the operation and maintenance layer, responsible for real-time monitoring of the platform’s operating status, and timely detection and handling of abnormal situations. Common monitoring systems include:

• Prometheus: An open-source system monitoring and alerting tool, suitable for monitoring various metric data. Official website: https://prometheus.io/
• Grafana: An open-source visualization tool that can be integrated with monitoring systems such as Prometheus, providing rich charts and dashboards. Official website: https://grafana.com/

Note: Prometheus monitoring requires relevant settings in the application, otherwise monitoring will not be possible.

Backup and Recovery

Backup and recovery are important safeguards of the operation and maintenance layer, ensuring that the system can be quickly restored in the event of a failure. Common backup and recovery strategies include:

• Full backup: Regularly perform full backups of the system to ensure data integrity.
• Incremental backup: Perform incremental backups of changed data to improve backup efficiency.
• Disaster recovery: Develop a disaster recovery plan to ensure that the system can be quickly restored in the event of a major failure.

Note: All backup solutions need to be handled on the program side, which may require additional development.