Azure Storage 1.9.2

tags: python | azure blob | cloud storage | distributed access

Description

Azure Storage includes object, file, disk, queue, and table storage. There are also services for hybrid storage solutions, and services to transfer, share, and back up data.

CDC Command Lines

* **extend_schemas_configuration_cli**
This CDC CLI is used to replace/upload files from the CDC to Azure Blob. Input files must be in ".yaml" format only. Example: extend_schemas_configuration_cli --schemas_config_file_name=extend_config.yaml

* **get_schemas_details_cli**
This command line is used to fetch observables schema details. Example: get_schemas_details_cli --download_file=true --product=["ms_sentinel"] --type=["base_events"] --name=["device"] --number=2

* **replace_schemas_configuration_cli**
This CDC CLI is used to replace/upload files from the CDC to Azure Blob. Input files must be in ".yaml" format only. Example: replace_schemas_configuration_cli --schemas_config_file_name=config.yaml

Workflows

* **extend_schemas_configuration**
This CDC CLI is used to replace/upload files from the CDC to Azure Blob. Input files must be in ".yaml" format only.

* **get_file_content_subworkflow**
This workflow is used to get the content of yaml file attached to a CDC incident, alert, or channel.

* **get_schemas_details**
This workflow is used for getting details about observables schema.

* **replace_schemas_configuration**
This CDC CLI is used to replace/upload files from the CDC to Azure Blob. Input files must be in ".yaml" format only.

Rules

No rules

Sensors

No sensors

Triggers

No triggers

Observable Schema

REPLACE CLI

• Replace observables schema configuration in Azure Blob:

Instructions:

• Write the 'yaml' file which contains observables extraction schema. See the information about the schema structure in ms_sentinel integration.
• Drop the 'yaml' file to the relevant CDC chat.
  Available options:
• schemas_config_file_name (Mandatory) - name of the 'yaml' file to replace.

Example:

replace_schemas_configuration_cli --schemas_config_file_name=config.yaml

Algorithm flow:

• Read the 'yaml' file content from the CDC chat.
• Save the content to Azure Blob.
  When failed - fails and notifies the user.
  When succeeded - user gets a notification.
• Save the content to Redis cache.
  When failed - fails and notifies the user.
  When succeeded - notifies the user.

Validations:

• The following validations are done during observables schema extraction. In case one of them
  fails, an error will be thrown and a notification will be sent to the user:
• Validate yaml file.
• Validate schema structure.
• Validate that there are no duplicated records.

EXTEND CLI

Instructions:

• Write the 'yaml' file which contains observables extraction schema. See information about the schema
  structure in ms_sentinel integration.
• Drop the 'yaml' file to the relevant CDC chat.
  Available options:
• schemas_config_file_name (Mandatory) - name of the 'yaml' file to replace.

Example:

extend_schemas_configuration_cli --schemas_config_file_name=extend_config.yaml

Algorithm Flow:

• Read the file content from the CDC chat.
• Read the original file from Azure Blob. If not found, fails and notifies the user.
• Perform extension logic using the original file from Azure Blob and the new extended data schema.
• Save the new merged content to Azure Blob.
  When fails - fails and notifies the user.
  When succeeded - notifies the user.
• Save the new merged content to Redis cached - fails and notifies the user.
  When fails - fails and notifies the user.
  When succeeded - notifies the user.

Validations:

• The following validations are done during observables schema extraction. In case one of them
  fails - an error will be thrown and a notification will be sent to the user:
• Validate the yaml file.
• Validate the schema structure.
• Validate that there are no duplicated records.
• Validate that there are no duplicated records between the original file to the extended file.

GET CLI

• Fetch the observables schema configuration:
• Available options:
• download_files (Optional) - When True - download the file to the CDC chat.
• product (Optional)- filter the schema according to 'product'. The parameter is a list of 'products' that
  enable filtering schema records according to their product.
• type (Optional) - filter schema according to 'type'. Parameter is a list of 'types' that enable filtering schema records according to their type.
• name - filter schema according to 'name'. Parameter is a list of 'types' that enable filtering schema records according to their type.
• number (Optional) - returns only several records from a schema result.

Example:

get_schemas_details_cli --download_file=true --product=["ms_sentinel"] --type=["base_events"] --name=["device"] --number=2

Algorithm Flow:

• Tries to extract observables schema information from Redis.
• If the schema was not found in Redis - search in Azure Blob.
• If the schema was not found in Azure Blob - fails and notifies the user.
• If the schema was found in Azure Blob - returns it to the user in 'generic_post' adaptive card and/or downloads the file according to user configuration.

Known Issues

No known issues