Unomaly continuously analyzes streaming data produced by software systems (servers, services, applications, containers, and so on) to learn their normal behavior and detect anomalies. This analysis is done without the need for specific parsers or predefined knowledge of the log format or structure. As long as you can get the data into Unomaly, it can be analyzed. You don’t need to select which types of data to send to Unomaly–the more data that you send to it, the better the picture it forms of your systems and the better it is able to detect problems within your environment.

Data input options

Getting data into Unomaly means configuring the software systems to send their data to the Unomaly standalone or worker instance. You can use log shippers, standard data protocols, such as syslog, or one of our pre-built integrations to collect data from other technologies.

Unomaly supports log messages at a maximum of 8192 bytes, including protocol headers. If a larger log message comes into Unomaly then Unomaly will not analyze the message beyond the first 8192 bytes.

Ways to ingest logs

There are many standard agents that specialize in collecting log data and forwarding them to another platform for processing. Unomaly supports data inputs from Fluentd, Logstash, and Graylog.

Many systems have built-in support for sending their data directly via standard communication protocols. Unomaly supports Syslog, SNMP, and HTTP.

Forward logs to Unomaly

If you already use other data technologies or log management tools to collect and process your data, you can also forward that data to Unomaly for analysis by using pre-built integrations. For example, Unomaly provides integrations to analyze data from Docker, Splunk, AWS CloudWatch logs, and many others.

Common input sources include log data from Unix and Linux, Windows Event Logs, and VMWare ESX.

What happens when Unomaly receives the data?

Once the instance is receiving the data it will first start tokenizing it to create a structural representation that can be used by the learning algorithms. Unomaly has a universal tokenizer that works for virtually all log data.

During this learning process, Unomaly continuously analyzes the generated data, creates profiles for the system, and updates the learnings database. These profiles capture the parameters and frequencies of the events to determine the baseline, or normal behavior, of each system. Read more about “How Unomaly learns behavior”.

How can you tell if the data input works?

When the data inputs are configured properly and Unomaly is receiving them, they will show up in Unomaly as systems and the data will be correctly tokenized, and identified as normal or anomalous events.

If new systems do not show up in the interface, Unomaly may not be configured to receive the data. You can troubleshoot this issue by checking that the communications settings are configured to receive data and that the services that handle ingestion and queuing of the data are running. See Troubleshooting data ingestion.

If Unomaly successfully receives the data, but the data does not look correct or as you expect it to, it may be an issue with the tokenization of the data. For example, if too many events with very long log lines are being recognized as the same type of events, you can increase the number of tokens that Unomaly will check for in an event. See Troubleshooting tokenization issues.

Next steps

Now that you have added data to Unomaly, learn more about systems and how to organize systems into groups.