Posts by chrissmith

    chrissmith added a new solution:

    Quote

    Overview

    The MISP DXL Python client library provides a high level wrapper for invoking the MISP REST APIs via the Data Exchange Layer(DXL) fabric.

    This client requires the MISP DXL Service to be running and available on the DXL fabric.

    Documentation

    See the Wiki for an overview of the MISP DXL Python Client Library and usage examples.


    See the MISP DXL Python Client Library documentation for installation instructions, API documentation, and usage examples.

    chrissmith added a new solution:

    Quote

    Overview

    The MISP DXL Python Service exposes access to the MISP REST APIs via the Data Exchange Layer (DXL) fabric. The service also provides support for forwarding MISP ZeroMQ message notifications to the DXL fabric.

    Documentation

    See the Wiki for an overview of the MISP DXL Python Service and usage examples.


    See the MISP DXL Python Service documentation for installation instructions, API documentation, and usage examples.

    Great question.


    The ability to restart Docker containers is a feature of Docker itself, not anything OpenDXL related. The following page discusses the various options that can be used when instantiating a container:


    https://docs.docker.com/config…containers-automatically/


    As an example, the following command could be used to instantiate a VirusTotal container that would always restart:


    docker run -d –-name dxlvtapiservice --restart always -v /home/myuser/dxlvtapiservice-config:/opt/dxlvtapiservice-config opendxl/opendxl-virustotal-service-python


    Thanks,

    Chris

    I would to see if I can send ePO a message or PC name and IP and see if it can trigger an event like a system scan.

    At this time, the best way to interact with ePO with an OpenDXL client is via the ePO DXL Service.


    To simplify the deployment of this service, a Docker container is available that can be quickly instantiated with Kitematic (see the Getting Started With OpenDXL Video Series) .


    Once installed and configured (see Service Configuration) you can use the OpenDXL Console to send requests to the service and ultimately to ePO via DXL.


    Please let us know if you need any further help configuring the service, using the OpenDXL Console, or sending requests to ePO.


    Thanks,

    Chris

    Hi-


    There are a couple of ways to send an event.


    You could send an event via an OpenDXL Client (Python, etc.). For example:



    Another way without any coding required would be to use the OpenDXL Console. The following series of videos walks through the steps to configure an OpenDXL environment, including the OpenDXL Console.


    Getting Started With OpenDXL Video Series


    DXL allows you to create your own topics. Some topics that are registered by products require authorization. You mention wanting to see the event in ePO. What is the use case you are trying to achieve by sending the event?


    Thanks a lot,

    Chris

    Great question!


    DXL intentionally does not dictate the format of the payload. This allows for a wide variety of integrations to be published on the fabric.


    Currently, the vast majority of solutions utilize JSON, but there are some that use XML, and others that use raw binary.


    With that said, for a particular sets of topics, it absolutely make sense to standardize the payloads. As you mention standardizing on the topics that utilize STIX and CEF makes sense. We have discussed adding a new section to this site that documents the "catalog" of topics and related formats that are available on DXL. I think something like this will be necessary as the number of integrations available on DXL continues to grow.


    Hope this helps,

    Chris


    Those errors indicate that the client is unable to connect to some of the broker(s) that are listed in the dxlclient.config file. Many times those errors can be ignored if the client is ultimately able to connect to one of the brokers listed.


    Do the OpenDXL client samples run successfully?


    Thanks,

    Chris

    I am not very informed on this particular integration, but based on the error message and the fact that it is attempting to set a TIE reputation, my guess is that it is an authorization issue.


    The client that is running this script must be authorized to set TIE reputations.


    The FAQ for the TIE DXL Python Client Library contains the following:

    Q: I receive a timeout, "dxlclient.exceptions.WaitTimeoutException: Timeout waiting for response to message", when attempting to set the reputation of a file/certificate


    A: This typically occurs due to the Python client not having permission to send messages to the /mcafee/service/tie/file/reputation/set topic (for files) and the /mcafee/service/tie/cert/reputation/set topic (for certificates).


    The following page provides an example of authorizing a Python client to send messages to an authorization group. While the example is based on McAfee Active Response (MAR), the instructions are the same with the exception of swapping the TIE Server Set Enterprise Reputation authorization group in place of Active Response Server API:


    https://opendxl.github.io/open…on/pydoc/marsendauth.html


    Hope this helps,

    Chris

    Great question.


    At this point (as you mentioned), the authentication to the fabric is certificate-based. Once a connection has been established, certificates or tags (if ePO-managed) are used to control what the particular connection can send or receive.


    The identity on the fabric is the connection identifier. In ePO-managed fabrics this is the GUID of the McAfee agent that the connection was made on behalf of.


    There have been a number of discussions regarding the enhancement of the fabric to provide a richer set of authorization models (use of tokens, etc.).


    With the fabric as it stands today, you can develop your own session-based interaction model with services. The following thread post contains details about invoking the same service instance on the fabric (versus round-robin semantics):


    Discussion Thread: Multiple services?


    With the type on invocation model described in the link above, the service could provide its own authentication model, sessions, etc.


    Thanks,

    Chris

    You need to build the distribution.


    To do that, you need to run the following command in the cloned source directory:


    python dist.py


    You also need to have Sphinx installed for the documentation generation. Thus, prior to running the above command install Sphinx via PIP:


    pip install sphinx


    Thanks,

    Chris

    Hi-


    A developer on the DXL team created a branch some time ago and made significant progress toward adding Python 3 support. I will talk to him and see if he can update it to the latest code and post back to the GitHub repository. Not sure what his current schedule is like, so it may take a bit for him to post it.


    Thanks,
    Chris

    Hi Thorsten-


    I configured an environment with DXL Broker 3.1.0.595 and confirmed that I was able to set a TIE reputation.


    A couple of things you can look for if you are still having issues.

    1.) Check the broker log file to see if authorization errors are occurring.

    The broker log file is named dxlbroker.log and can be found under the /var directory on the broker. If you see an error similar to the following when attempting to set a reputation in TIE, the authorization is not property configured:


    [139640851416928] 08/04/17 16:52:44 [I]  Not authorized for send: 9067752a-6f80-476d-848b-615a26375431 (/mcafee/service/tie/file/reputation/set) (message/handler/src/AuthorizationHandler.cpp:44)

    2.) Confirm that the certificate is found in the broker's authorization policy

    If the authorization policy on the broker does not contain two entries (ePO and your certificate) that indicates that the policy has not been updated correctly on the broker. Check the contents of the topicauth.policy also found under the /var directory on the broker.


    If the "file reputation" topic within the PUBLISHERS section only has one entry (as shown below) that indicates the policy has not been received correctly by the broker:


    /mcafee/service/tie/file/reputation/set={da34e96a-1d00-47c1-ba54-6927852af1af}


    A correct policy which contains ePO and the certificate should appear similar to the following:


    /mcafee/service/tie/file/reputation/set=d86e4f73a454623b2fe03a5298728e8c1a9e4f7f;{da34e96a-1d00-47c1-ba54-6927852af1af}


    Hope this helps,

    Chris

    Are there plans to integrate the service conversation model into the client?


    There are no current plans to integrate that pattern, but it is definitely something we could look at in the future. Also, if anyone is feeling ambitious, please fork the client and submit a pull request. :)


    It looks like the service is still selected round robin. Is there a way to select the service to call? For example I may have multiple versions of a product on the fabric. Is it possible to select the one to use?


    You are correct. In my previous code example, the initial service was selected via round-robin. That service was used for all subsequent requests (by obtaining the service_id from the initial response).


    If you would like to invoke a specific instance of a service, you can use the "service registry". The topic /mcafee/service/dxl/svcregistry/query is used to query the service registry. The ability to query the registry is pretty limited at this point. If you pass an empty JSON object you will receive information on all of the services registered. If you specify a serviceType in the JSON object you will receive information on all services of that particular type. For example, you could query for all Threat Intelligence Exchange (TIE) services.


    The following code example registers 3 different services. The first two services are of the same type ("myService"), while the third service is a different type ("anotherService"). The code then performs two different queries on the service registry. The first query returns all services, while the second query returns services of the type, "myService".



    The output for the first query should appear similar to the following:



    The output for the second query should appear similar to the following:



    As you can see there is quite a bit of information on each service. It is also important to note that the service registration information includes a serviceGuid. This value can be specified in the service_id of a request message to invoke that particular service. Other useful information includes metaData which could be used as a means of selecting the service to invoke.


    Hope this helps,

    Chris

    Service zones do not apply any restrictions. Their purpose is merely to route service requests to services that are available within the current zone. If a service is not found in that zone, it will look in a parent zone. If multiple services exist in the same zone, they will round-robin.


    You can create conversation-based clients by setting the service_id on the Request message. This will route the message to a specific service instance. The typical pattern is to send the initial request without setting the service identifier. In the initial case, the broker will select the service to invoke. After the initial invocation, the service_id as obtained in the initial Response message is used for all subsequent requests.


    I updated the service example that is included with the OpenDXL Python Client SDK to create multiple services as show below:



    The output from this sample should look similar to the following:


    Python: Multiple Service Output
    1. Service received request payload: ping
    2. Client received response payload: pong from service {1c456f55-e6d2-4b4f-95e2-9011cc609d3b}
    3. Service received request payload: ping
    4. Client received response payload: pong from service {2108df2b-cde1-4773-bbb1-7e8443f44ade}
    5. Service received request payload: ping
    6. Client received response payload: pong from service {78aea342-c740-404e-83fc-6fb2ae6ecd61}
    7. Service received request payload: ping
    8. Client received response payload: pong from service {a5538f3d-d14a-4ccf-92d5-e9e3049e2b27}
    9. Service received request payload: ping
    10. Client received response payload: pong from service {4388a0f8-4908-45e6-af02-eae461602141}


    As show above, all of the service instances are invoked in a round-robin fashion. I then updated the invoking client code portion (show below) to capture the service_id from the initial request and use it on all subsequent requests.




    Once those changes have been made, the output should look similar to the following:


    Python: Single Service Invoked Output
    1. Service received request payload: ping
    2. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    3. Service received request payload: ping
    4. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    5. Service received request payload: ping
    6. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    7. Service received request payload: ping
    8. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    9. Service received request payload: ping
    10. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}



    As shown above, the same service is invoked even though multiple services are registered with the fabric.


    Hope this helps,

    Chris

    Thanks a lot for collecting that information.


    Support for certificate-based authorization was added in 3.0.1. For it to work correctly, all brokers in the fabric must be at least on that version. In the fabric displayed, only one broker is at that version (3.1.0.595). The other brokers embedded in the TIE servers are still on a 2.x version.


    Our testing of 3.1.0.595 did include a suite of tests related to certificate based authorization. However, I will configure a broker today matching that version to confirm it is working as expected.


    Thanks,

    Chris