Интеграции со сторонними системами

Для интеграции со сторонними системами можно использовать API/gRPC/прямые запросы в базу.

Прямой запрос в postgresql

Количество подключенных источников (целое число):
-- SELECT
--     COUNT(*)::int
-- FROM preferences.collectors
-- WHERE preferences.collectors.enabled = True
Количество запущенных директив (целое число):
-- SELECT
--     COUNT(*)::int
--          FROM corrdisp.realtime_correlators
-- WHERE corrdisp.realtime_correlators.active = True
Есть id инцидента, необходимо получить все поля из инцидента из каждой колонки:
-- SELECT
--     id,
--     directive_id,
--     assigned_to,
--     is_retro,
--     initial_time,
--     registration_time,
--     close_time,
--     status,
--     severity,
--     event_doc_keys,
--     gossopka_sending_status,
--     security_label,
--     tenant_id,
--     correlator_ids,
--     recommendation,
--     has_errors,
--     updated_at,
--     comments,
--     status_reason,
--     histories,
--     gossopka_incident_type,
--     gossopka_incident_id,
--     response_stage,
--     description,
--     asset_ips,
--     threats_array,
--     gossopka_incident_category
-- FROM incman.incidents
-- WHERE id = 16
Все поля по инциденту и директиве:
-- SELECT
--     id,
-- -- directives START
--     directive_id,
--     name,
--     directive_severity,
--     directive_assigned_to,
--     reaction_script_filename,
--     reaction_is_enabled,
--     created_at,
--     directive_updated_at,
--     aggregate_in,
--     aggregated_min_count,
--     group_id,
--     gossopka,
--     directive_recommendation,
--     aggregate_greedily_count,
--     directive_gossopka_incident_type,
--     rule,
--     deleted_at,
--     risk_score,
--     author,
--     reference_url,
--     note,
--     timestamp_override,
--     rule_edit_state,
--     produce_incident,
--     produce_aggregated_event,
--     store_keys_in_event,
--     directive_threats_array,
--     directive_gossopka_incident_category,
-- --     END
--     assigned_to,
--     is_retro,
--     initial_time,
--     registration_time,
--     close_time,
--     status,
--     severity,
--     event_doc_keys,
--     gossopka_sending_status,
--     security_label,
--     tenant_id,
--     correlator_ids,
--     recommendation,
--     has_errors,
--     updated_at,
--     comments,
--     status_reason,
--     histories,
--     gossopka_incident_type,
--     gossopka_incident_id,
--     response_stage,
--     description,
--     asset_ips,
--     threats_array,
--     gossopka_incident_category
-- FROM incman.incidents
--     JOIN (
--         SELECT
--             id as dir_id,
--             name,
--             severity as directive_severity,
--             assigned_to as directive_assigned_to,
--             reaction_script_filename,
--             reaction_is_enabled,
--             created_at,
--             updated_at as directive_updated_at,
--             aggregate_in,
--             aggregated_min_count,
--             group_id,
--             gossopka,
--             recommendation as directive_recommendation,
--             aggregate_greedily_count,
--             gossopka_incident_type as directive_gossopka_incident_type,
--             rule,
--             deleted_at,
--             risk_score,
--             author,
--             reference_url,
--             note,
--             timestamp_override,
--             rule_edit_state,
--             produce_incident,
--             produce_aggregated_event,
--             store_keys_in_event,
--             threats_array as directive_threats_array,
--             gossopka_incident_category as directive_gossopka_incident_category
--         FROM corrdisp.directives
--     ) AS directive
--     ON directive_id = directive.dir_id
-- WHERE incman.incidents.id = 16

Прямой запрос в ClickHouse

Количество событий в секунду (целое число, необходимо обновлять раз в 5 сек)

-- SELECT
--     COUNT(*)
-- FROM komrad_events.events
-- WHERE dateDiff('second', events.w_time, now()) <= 1

Среднее значение за час в секунду

-- SELECT
--     COUNT(*) / (1 * 60 * 60)
-- FROM komrad_events.events
-- WHERE dateDiff('second', events.w_time, now()) <= 1 * 60 * 60

Число событий в минуту (epm), количество событий по протоколам

SELECT
    'total_events_count_by_collector_type_last_24h' AS name,
    count()  as value,
    'count events by collector type last 24 hours' AS help,
    map('collector_type', collector_type) AS labels,
    'counter' AS type
FROM events
WHERE key_time >= timestamp_sub(HOUR, 24, now())
GROUP BY collector_type

SELECT
    'total_events_count_by_collector_type_last_hour' AS name,
    count()  as value,
    'count events by collector type last hour' AS help,
    map('collector_type', collector_type) AS labels,
    'counter' AS type
FROM events
WHERE key_time >= timestamp_sub(HOUR, 1, now())
GROUP BY collector_type

Запросы в формате Prometheus

SELECT * FROM  komrad_events.view_widgets FORMAT Prometheus

Число событий в минуту (epm), количество событий по протоколам в prometheus формате для отображения в Grafana

CREATE OR REPLACE VIEW komrad_events.view_widgets
            (
             `name` String,
             `value` Float64,
             `help` String,
             `labels` Map(String, String),
             `type` String
                ) AS
SELECT names as name, value, help, labels, type from (
                                                         SELECT
                                                             'total_events_count' AS names,
                                                             total_rows  as value,
                                                             'total IS events' AS help,
                                                             map('total_events_count', '') AS labels,
                                                             'counter' AS type
                                                         FROM system.tables
                                                         WHERE database = 'komrad_events' AND tables.name = 'events'
                                                         )
UNION ALL
SELECT
    'total_events_count_by_collector_type_last_24h' AS name,
    count()  as value,
    'count events by collector type last 24 hours' AS help,
    map('collector_type', collector_type) AS labels,
    'counter' AS type
FROM events
WHERE key_time >= timestamp_sub(HOUR, 24, now())
GROUP BY collector_type
UNION ALL
SELECT
    'total_events_count_by_collector_type_last_hour' AS name,
    count()  as value,
    'count events by collector type last hour' AS help,
    map('collector_type', collector_type) AS labels,
    'counter' AS type
FROM events
WHERE key_time >= timestamp_sub(HOUR, 1, now())
GROUP BY collector_type
UNION ALL
SELECT
    'asset_ip_count' AS name,
    count()  as value,
    'top asset_ips count last 24 hours' AS help,
    map('asset_ip',  arrayJoin(asset_ips)) AS labels,
    'counter' AS type
FROM events
WHERE key_time >= timestamp_sub(HOUR, 24, now())
GROUP BY arrayJoin(asset_ips)
ORDER BY value DESC
LIMIT 10

API

Авторизация

Для авторизации необходимы:

  • Логин

  • Пароль

  • Корневой сертификат

Получение токена
  • Логин и сохранение токена (куки) в файл cookie_file.txt

curl --cacert ./ca.pem -i -H "Accept: application/json" -H "Content-Type: application/json" --data '{"Login":"admin","Password":"admin"}' -c cookie_file.txt https://[ip_komrad]/api/v1/login
  • Проверка сессии по файлу с токеном cookie_file.txt

curl --cacert ./ca.pem -i -H "Accept: application/json" -H "Content-Type: application/json" -X GET -b cookie_file.txt https://[ip_komrad]/api/v1/me
  • Пример получения данных по инциденту с использованием файла с токеном

curl --cacert ./ca.pem -i -H "Accept: application/json" -H "Content-Type: application/json" -X GET -b cookie_file.txt https://[ip_komrad]/api/v1/incidents/{\ID\}
Проводите периодические проверки доступности сессии, время сессии указывается в конфигурации pauth-server.

Запросы

Инциденты:

Список инцидентов: /api/incidents (с пагинацией и фильтрацией) - можно получить все в списке, включая:

  • Тип инцидента (Status) /api/incidents/{ID}

  • Критичность инцидента (Severity) /api/incidents/{ID}

  • Статус реагирования (В работе/завершено и т.д.) - ГосСОПКА (GosSOPKASendingStatus) /api/incidents/{ID}

События:

События за период: /api/filtered-events?$sort=GenerationTime&$order=desc&$page=1&$search=&$limit=25&$format=list&From=2023-02-01T11:13:00.000Z&To=2023-02-01T13:13:00.000Z&Query=&CustomFields=CollectorID,GenerationTime,Raw,CollectorType

Технические сведения (IP, доменное имя, url, порт, протокол) (incman-incidents: asset_ips) /api/filtered-events/{EventKey}

Технические сведения о вредоносной системе (IP, доменное имя, url, порт, протокол, описание используемой уязвимости) /api/filtered-events/{EventKey}

Источники

Количество подключенных источников (целое число) /api/scanner-assets с пагинацией

Пример:

curl --cacert ./ca.pem "Accept: application/json" -H "Content-Type: application/json" -X GET -b  'https://[ip_komrad]/api/scanner-assets?$sort=CreatedAt&$order=desc&$page=1&$limit=25&$format=list'

Директивы

Количество запущенных директив(целое число) ( /api/directives с пагинацией и фильтрация (DB:realtime_correlators))

curl -X 'GET' 'https://[ip_komrad]/api/directives?$page=1&$limit=25&$format=list' --cacert ./ca.pem -b cookie_file.txt -v