These options are accessible only via users with respective administration privileges. (See User management)
Screenshots
Administration dashboard showing statistics and all its functionalities (sidebar)
This is the multi-page printable view of this section. Click here to print.
These options are accessible only via users with respective administration privileges. (See User management)
Administration dashboard showing statistics and all its functionalities (sidebar)
Organizations can be added with following parameters defining them: name, description, address.
Define roles with certain permissions.
Manage users and assign them a role, permissions and an organization.
Add new Organization
Edit basic user role
Add new User
To get started there is list of RSS sources we worked with: Initial setup_
Word lists can have the following functionalities (displayed under “usage”):
To activate include or exclude lists, they need to be added to the default source group.
It has to be mentioned, that this include/exclude filtering happens during the news item collection. Therefore, only filtered news items will be stored in the database and displayed in “Assess”.
After the collection, it is possible to adapt news items.
Therefore, following bots are currently available:
CRUD: Bots can be created, updated and deleted.
Index: Decides the order of bots
RUN_AFTER_COLLECTOR: Indicates if bot is active after collection
After all settings are made, sources can be collected. Either collect all sources by clicking on the “collect sources” button, or collect single sources.
Sources for gathering data are set in the OSINTSources. It is possible to:
Select Import and choose desired JSON file for import. (See Initial Setup)
Select Export to download a JSON file containing your established collectors.
Select Collect Sources to aggregate information from all established OSINT sources.
When creating new Reports, one of the created report types have to be selected (see Analyze).
Desired attributes need to be created first. Then they can be managed by the admin user. Besides name, description and default value also type, validator and validator parameter can be set.
Add new Attribute
Report Types - Create new Report Type
Report Types - Add new Attribute Group
Report Types - Select new Attribute from list
The administration view now allows users to use the Preview feature to see the result of the configuration without the items being processed further for the Assess view. This feature is available for RSS, Simple Web and RT collector.
RSS Collector enables Taranis AI to collect data from a user-defined RSS feed (See RSS feeds details).
json
] (can be used to add additional headers, not all headers work as expected)Summary
field of RSS feed)The RSS Collector supports the use of XPath for locating elements. (See Simple Web Collector Advanced configuration)
{ "AUTHORIZATION": "Bearer Token1234", "X-API-KEY": "12345", "Cookie": "firstcookie=1234; second-cookie=4321", }
Simple Web Collector enables Taranis AI to collect data using web URLs and XPaths.
The simplest way to use this collector is to use the WEB_URL field only. By using only the WEB_URL field, Taranis-AI autonomously determines the content to be collected. Even though it is mostly reliable, sometimes it is not perfect.
When content cannot be reliably collected using the Basic configuration, adding the attribute XPATH (See tutorial how to find it), can be useful. It is crucial to specify the XPath of the precise element containing the desired data.
To set up an RSS Collector for collecting posts from a Mastodon hashtag or user, follow these steps:
Finding the Mastodon RSS Feed URL:
.rss
to the hashtag URL. For example, to collect posts tagged with #cybersecurity:
https://mastodon.social/tags/cybersecurity.rss
.rss
to the user’s profile URL. Example:
https://mastodon.social/@username.rss
Creating a New RSS Source with Required Parameters: When creating the new RSS source, configure it with the following parameters. Here’s an example of how to fill out the fields:
https://mastodon.social/tags/cybersecurity.rss
)."summary"
to specify the main content location within each RSS entry."false"
since we’re not splitting entries into multiple items.Extend your compose.yml with a tor service, e.g.
tor:
image: "docker.io/dperson/torproxy:latest"
deploy:
restart_policy:
condition: always
environment:
# LOCATION: "AT"
logging:
driver: "json-file"
options:
max-size: "200k"
max-file: "10"
Read details about the used docker image here
The important setting is “PROXY_SERVER” in the OSINT Source you want to crawl.
RT Collector enables Taranis AI to collect data from a user-defined Request Tracker instance.
RT Collector collects tickets, translates all ticket attachments into individual News Items. A ticket is represented via a Story. It also collects ticket Custom Fields and saves it as key-value pairs represented with Story attributes, visible whilst Story editing. On each collector execution an update to existing Stories occurs, so editing the Story values in Taranis AI is not recommended and handling it more like read-only items is better.
Required fields:
http://localhost
).Optional fields:
Until the definitions of our MISP Objects are not officially part of the MISP platform, feel free to import them manually (see MISP Objects). This allows to edit the information of News Items and Story data directly in the MISP instance without Taranis AI.
MISP Collector enables Taranis AI to collect MISP events.
Required fields:
Optional fields:
Essentially it works exactly like other collectors with one exception: conflicts. Given the nature of the collaborative environment of MISP events (they can be changed in the MISP platform by the owning organisation and secondary organisations can submit change requests using the MISP proposals). Due to that, there will likely occur conflicts when attempting to update existing Stories that were, in the meantime, internally modified.
Generally, conflicts occur the moment, a Story is modified internally, and has not been pushed to MISP immediately. Therefore, it is recommended to always try to keep Stories in sync with the MISP events. To update them in MISP with the Story (see Connectors).
Conflicts can currently be resolved in the Connectors section accessible in the general and administration dashboard.
In the Conflict Resolution View it is possible to resolve given conflicts.
In the view, all stories that have conflicts will be shown in expandable cards. One by one it is possible to inspect them and resolve them.
At the top, the number of events (among the ones in conflict) have proposals to resolve is shown (only owned by your MISP organisation). It is recommended to resolve them first in MISP, before you resolve them in Taranis AI, so you work with the latest information. Once teh conflicts are resolved and changes are submitted, the Story will be updated without raising conflicts later on (until the Story gets modified internally). It is the users’ decision, whether they want to resolve the conflicts immediately with the version collected without proposals, or are skipped, preposals in MISP are resolved, and resolve conflicts later on next collection. In case a Story has proposals, it is possible to open the event in MISP with the “View Proposal” button.
The content on the left is the current Story. The content on the right hand side editor is the new content and is the content, which is the content that is used for the eventual update. It is possible to take the right or left side of each difference shown using the diff window by clicking the arrows on the left sides of each content window.
Both sides are freely editable and keyboard shortcuts for back and forward are supported. The context for keyboard shortcuts is decided based on where the cursor is (where it was clicked the last time - right or left side).
It is important, that the content on the right side stays a valid Story JSON.
With “Get Right Side” button, it is possible to check what content will be submitted. “Submit Resolution” button is used to submit the update.
Conflicts are stored in a temporary memory, to encourage a fresh MISP Collector recollection, before resolving conflicts and to prevent conflict resolution with outdated data.
Digest Splitting is a feature that allows the user to split all available URLs in the located element into individual News Items. The Digest Splitting Limit is the maximum number of URLs that will be split into individual News Items. If the limit is reached, the remaining URLs are dropped. The Digest Splitting Limit is set to 30 News Items by default but can be adjusted by the administrator. Useful in case of timeouts during collection of too many News Items.
Collectors will fail if the web page content is only available with JavaScript. In that case it is possible to turn on the Browser Mode. All requests will have JavaScript enabled, therefore, it is slower and can use more resources.
Supported options:
The MISP connector is currently in an experimental stage. Please submit issues when you discover any problems or need support with its usage.
Until the definitions of our MISP Objects are not officially part of the MISP platform, feel free to import them manually (see MISP Objects). This enables you to edit the objects of News Items and Story data directly in you MISP instances without Taranis AI.
MISP Connector enables Taranis AI to push Stories to MISP in the representation of MISP Events.
The created events contain automatically an Event Report based on the Story content (currently it encompasses the Story description and Story summary, if you feel the Event Report would benefit extending with more information, please open a feature request for it here).
First the user with administrative privileges needs to make sure, the permissions connected to Connectors usage are assigned appropriately, by default, they are not assigned to any user, not even the admins themselves. Then the Connector needs to be setup in the Admin section under the Connectors tab.
Explanation of individual permissions:
Required parameters:
Optional parameters:
To send a story to MISP, use the button “Share to Connector” in Story options in Assess. When an update to the Story is later on made, it is recommended to share the change immediately, you need to do this manually like the first time.
Before a Story is pushed repeatedly to update the MISP event, it is important to check if any proposals are pending in the MISP event, as it would override the original content of it and would make it difficult to resolve the proposals correctly.
When a Story is collected, that is not owned by your configured organisation, it is possible to update it in Taranis AI and by using the same “Share to Connector” button in the Story options a proposal to the event is created. Then the owning organisation should review this proposal, and approve or reject the changes.
The Email Publisher allows sending out Products.
Note: The EMAIL_SENDER and EMAIL_RECIPIENT parameters are used to construct the message envelope used by the transport agents. Message headers are not modified by these parameters in any way.
Required fields are marked with a *.
Once the publisher is created, it becomes available in the “Publish” section of each product. To send out a product via email, the product must be “Rendered” first. To render a product, use the option available in the product’s view.
All crucial fields are editable, with the most important being Type, Template, and Report Types.
While there are several prebuilt product types available, users also have the option to create their own product types using custom templates.
It can be beneficial to create custom Product Types to meet desired results with the publishers.
This is an example to render arbitrary values and loop over attributes.
src/core/core/static/presenter_templates
,TITLE: {{ data.report_items[0].get('title') | default('No title provided', true) }}<br>
DATE CREATED: {{ data.report_items[0].get('created') | default('Not available', true) }}<br>
LAST UPDATED: {{ data.report_items[0].get('last_updated') | default('Not available', true) }}<br>
{% for name, attribute in data.report_items[0].get('attributes').items() %}
{{ name }}: {{ attribute }}<br>
{% endfor %}
If one is interested in creating own templates, it is a good to start to render the object {{ data }}
first, to understand how to parse the object properly.
It is also possible to copy src/core/core/static/presenter_templates/<new-custom-template.txt>
to a dynamic folder src/core/taranis_data/presenter_templates
so the restart is not necessary.
If needed, templates can be utilized for more complex renderings by leveraging custom attributes.
Currently, this functionality is demonstrated in the text_template.txt
file, where the attribute omission
of type “Omit Keys” allows for the exclusion of unnecessary attributes from publication. To employ this feature, the administrator simply needs to add this attribute to the relevant report type. Then, within a specific report (Analyze View), they can specify the attributes to omit by listing them as comma-separated strings.
It is essential to ensure that the “Name” used for the report type attribute matches exactly with the key used in the template.
The admin user can access the Taranis API through Swagger UI. Swagger UI displays OpenAPI specifications as an interactive API documentation.
see: Swagger UI
Taranis instance is alive