Query Engine 6.4.0
  • 01 Mar 2023
  • 4 Minutes to read
  • Dark
    Light
  • PDF

Query Engine 6.4.0

  • Dark
    Light
  • PDF

Article Summary

tags: python | Query Automation | SIEM Query engines


Description

Query Engine and Investigation Framework Automation is created on top of sensors connecting the CDC to various SIEM and EDRs such as Sentinel, Splunk, QRadar, Cybereason, ADX, etc. It supports CDC users by automating their ability to run queries on different SIEMs. These SIEMs are configured for the customer within the CDC itself.

As part of alert triage and investigation, analysts typically run predictable queries using SIEM and EDR, in response to specific alerts. Instead of automating these individual queries for them, CyberProof has developed a mechanism that allows CDC analysts to configure CDC to automatically run specific SIEM and EDR queries in response to specific alerts.

The query engine runs the query in response to an alert. To make this process simpler, we have created an alias for each query, and analysts can run the query by passing alias details in the CLI. We have enabled the upload of use cases and query files using CDC commands. These commands allow analysts to add/modify the queries/automations, with no additional effort required from the Integrations team.

The queries accept parameter values from the alert/observables. We use features of the CSV preview to showcase the output returned in CSV. This preview is shown on the CDC screen itself.

Integration Type:Automation
Information read:Alert/observable provided as parameter and corresponding SIEM/EDR queries to fetch the required results.
API Supported:All existing Sensors to communicate to SIEM and EDR.
Input:Alert/Observables and query alias details.
Output:Detailed output from SIEM/EDR, as per the specific queries run by the analyst. Or appropriate error message if no matching results/any error in execution.

Customer Configuration

No specific configuration required at the customer end.


CDC Command Lines

* **automate_query_execution_cli**
The CLI of the CDC, for manually triggering the automated query execution workflow in the CDC message thread. Note that Input 'debug_message' must be a boolean value (true/false).

OptionTypeDescriptionRequired
alert_idstringThe alert_id to work upon.False
debug_messagebooleanThe debug flag to display messages in chat; e.g., true/false.False

* **execute_query_using_alias_cli**
The CLI of the CDC, for manually executing queries using query_alias in the CDC message thread. Note that Input 'debug_message' must be a boolean value (true/false).

OptionTypeDescriptionRequired
query_aliasstringThe query_alias of the query to be executed.True
query_parametersobjectThe object to be replaced in the query to be executed.False
alert_idstringThe alert_id to work upon.False
debug_messagebooleanThe debug flag to display messages in chat; e.g., true/false.False

* **get_query_details_and_download_file_cli**
This CLI of the CDC is used to fetch use-case query details, as well as to enable the downloading of a file if selected. Note that Input 'download_file' must be a boolean value (true/false).

OptionTypeDescriptionRequired
download_filebooleanAttach the file in the original format.False

* **upload_config_cli**
This CLI of the CDC is used to upload files from CDC to Azure Blob. Note that at least one of use case config or rule config file name needs to be mentioned. Input files for this CLI should be mandatorily in ".yaml" format only. Input 'overwrite' must be a boolean value (true/false). While running the CLI, enter Incident ID or Channel ID or Alert ID (in CDC version < 2.2).

OptionTypeDescriptionRequired
alert_idstringAlert IDFalse
incident_idstringIncident IDFalse
channel_idstringChannel IDFalse
rules_config_file_namestringRules config file nameFalse
use_case_config_file_namestringUse case config file nameFalse
overwritebooleanOverwrite flag statusFalse

Workflows

* **automate_query_execution**
This workflow will be triggered when a new alert is created in the CDC.

* **automate_schedule_query_execution**
This workflow will be triggered from Logic Apps for running static scheduled queries.

* **check_pack_installed**
This is a sub-workflow for identifying packs' installed status.

* **execute_query**
This workflow preprocesses the query and maps parameters from config to the query definition.

* **execute_query_for_adx**
This workflow executes the query definition and returns the Redis key.

* **execute_query_for_ms_defender**
This workflow executes the query definition and returns the Redis key.

* **execute_query_for_qradar**
This workflow executes the query definition and returns the Redis key.

* **execute_query_for_sentinel**
This workflow executes the query definition and returns the Redis key.

* **execute_query_for_splunk**
This workflow executes the query definition and returns the Redis key.

* **execute_query_logic_apps**
Logic Apps action for executing queries using query_alias in the CDC.

* **execute_use_case_query**
This is a sub-workflow for an alert-created workflow.

* **fetch_blob_configs**
This workflow rules use_case abd chart config from Azure Storage, in parallel.

* **fetch_query_details**
This workflow reads the rule configuration from Azure Blob storage and fetches query details and performs additional checks.

* **get_cdc_version**
Get CDC version.

* **get_file_content_subworkflow**
This workflow is used to get the content of the yaml file attached to a CDC incident, alert, or channel.

* **get_query_details**
This workflow is used for getting details about a query.

* **identify_pack_per_query**
This is a sub-workflow for identifying packs' installed status.

* **post_message_cdc**
Post the message in the CDC, in the channel/alert/incident/thread.

* **prerequisites_fetch**
Fetch the prerequisites for the validation and execution of queries.

* **schedule_query**
Runs a scheduled query from Logic Apps.

* **upload_to_cdc**
Upload query data to the CDC.

* **upload_to_cdc_2**
Upload query data to the CDC, and display a vega chart under the graph section of the CDC (for 2.7 versions and above).

* **validate_alert_trigger_condition**
This workflow finds matching queries with specified trigger conditions.


Rules

* **new_alert_listener**
Triggering an automate_query_execution workflow when an alert is created in the CDC.


Sensors

No sensors


Triggers

No triggers


Policies

NameTypeDescriptionResourcePolicy ActionThreshold
automate_query_execution_policyaction.concurrencyLimits the concurrent executions for automate_query_executionquery_engine.automate_query_executioncancel20

Known Issues

No known issues


Was this article helpful?