Posts by chrissmith

  • how can I send a mock event to see what is received by ePO

    Quote from clasha:

    I would to see if I can send ePO a message or PC name and IP and see if it can trigger an event like a system scan.

    At this time, the best way to interact with ePO with an OpenDXL client is via the ePO DXL Service.


    To simplify the deployment of this service, a Docker container is available that can be quickly instantiated with Kitematic (see the Getting Started With OpenDXL Video Series) .


    Once installed and configured (see Service Configuration) you can use the OpenDXL Console to send requests to the service and ultimately to ePO via DXL.


    Please let us know if you need any further help configuring the service, using the OpenDXL Console, or sending requests to ePO.


    Thanks,

    Chris

  • how can I send a mock event to see what is received by ePO

    Hi-


    There are a couple of ways to send an event.


    You could send an event via an OpenDXL Client (Python, etc.). For example:



    Another way without any coding required would be to use the OpenDXL Console. The following series of videos walks through the steps to configure an OpenDXL environment, including the OpenDXL Console.


    Getting Started With OpenDXL Video Series


    DXL allows you to create your own topics. Some topics that are registered by products require authorization. You mention wanting to see the event in ePO. What is the use case you are trying to achieve by sending the event?


    Thanks a lot,

    Chris

  • DXL Event payload contents (or the absence of a payload standard)

    Great question!


    DXL intentionally does not dictate the format of the payload. This allows for a wide variety of integrations to be published on the fabric.


    Currently, the vast majority of solutions utilize JSON, but there are some that use XML, and others that use raw binary.


    With that said, for a particular sets of topics, it absolutely make sense to standardize the payloads. As you mention standardizing on the topics that utilize STIX and CEF makes sense. We have discussed adding a new section to this site that documents the "catalog" of topics and related formats that are available on DXL. I think something like this will be necessary as the number of integrations available on DXL continues to grow.


    Hope this helps,

    Chris

  • OpenDXL-WildFireTIE


    Those errors indicate that the client is unable to connect to some of the broker(s) that are listed in the dxlclient.config file. Many times those errors can be ignored if the client is ultimately able to connect to one of the brokers listed.


    Do the OpenDXL client samples run successfully?


    Thanks,

    Chris

  • OpenDXL-WildFireTIE

    I am not very informed on this particular integration, but based on the error message and the fact that it is attempting to set a TIE reputation, my guess is that it is an authorization issue.


    The client that is running this script must be authorized to set TIE reputations.


    The FAQ for the TIE DXL Python Client Library contains the following:

    Quote from TIE DXL Python Library FAQ:

    Q: I receive a timeout, "dxlclient.exceptions.WaitTimeoutException: Timeout waiting for response to message", when attempting to set the reputation of a file/certificate


    A: This typically occurs due to the Python client not having permission to send messages to the /mcafee/service/tie/file/reputation/set topic (for files) and the /mcafee/service/tie/cert/reputation/set topic (for certificates).


    The following page provides an example of authorizing a Python client to send messages to an authorization group. While the example is based on McAfee Active Response (MAR), the instructions are the same with the exception of swapping the TIE Server Set Enterprise Reputation authorization group in place of Active Response Server API:


    https://opendxl.github.io/open…on/pydoc/marsendauth.html


    Hope this helps,

    Chris

  • Open DXL Authenticatication

    Great question.


    At this point (as you mentioned), the authentication to the fabric is certificate-based. Once a connection has been established, certificates or tags (if ePO-managed) are used to control what the particular connection can send or receive.


    The identity on the fabric is the connection identifier. In ePO-managed fabrics this is the GUID of the McAfee agent that the connection was made on behalf of.


    There have been a number of discussions regarding the enhancement of the fabric to provide a richer set of authorization models (use of tokens, etc.).


    With the fabric as it stands today, you can develop your own session-based interaction model with services. The following thread post contains details about invoking the same service instance on the fabric (versus round-robin semantics):


    Discussion Thread: Multiple services?


    With the type on invocation model described in the link above, the service could provide its own authentication model, sessions, etc.


    Thanks,

    Chris

  • Documentation from source

    You need to build the distribution.


    To do that, you need to run the following command in the cloned source directory:


    python dist.py


    You also need to have Sphinx installed for the documentation generation. Thus, prior to running the above command install Sphinx via PIP:


    pip install sphinx


    Thanks,

    Chris

  • Python 3 support

    Hi-


    A developer on the DXL team created a branch some time ago and made significant progress toward adding Python 3 support. I will talk to him and see if he can update it to the latest code and post back to the GitHub repository. Not sure what his current schedule is like, so it may take a bit for him to post it.


    Thanks,
    Chris

  • Cannot change TIE Reputation when Certificate is used for DXL authentication

    Hi Thorsten-


    I configured an environment with DXL Broker 3.1.0.595 and confirmed that I was able to set a TIE reputation.


    A couple of things you can look for if you are still having issues.

    1.) Check the broker log file to see if authorization errors are occurring.

    The broker log file is named dxlbroker.log and can be found under the /var directory on the broker. If you see an error similar to the following when attempting to set a reputation in TIE, the authorization is not property configured:


    [139640851416928] 08/04/17 16:52:44 [I]  Not authorized for send: 9067752a-6f80-476d-848b-615a26375431 (/mcafee/service/tie/file/reputation/set) (message/handler/src/AuthorizationHandler.cpp:44)

    2.) Confirm that the certificate is found in the broker's authorization policy

    If the authorization policy on the broker does not contain two entries (ePO and your certificate) that indicates that the policy has not been updated correctly on the broker. Check the contents of the topicauth.policy also found under the /var directory on the broker.


    If the "file reputation" topic within the PUBLISHERS section only has one entry (as shown below) that indicates the policy has not been received correctly by the broker:


    /mcafee/service/tie/file/reputation/set={da34e96a-1d00-47c1-ba54-6927852af1af}


    A correct policy which contains ePO and the certificate should appear similar to the following:


    /mcafee/service/tie/file/reputation/set=d86e4f73a454623b2fe03a5298728e8c1a9e4f7f;{da34e96a-1d00-47c1-ba54-6927852af1af}


    Hope this helps,

    Chris

  • Multiple services?

    Quote from jdhoffman:

    Are there plans to integrate the service conversation model into the client?


    There are no current plans to integrate that pattern, but it is definitely something we could look at in the future. Also, if anyone is feeling ambitious, please fork the client and submit a pull request. :)


    Quote from jdhoffman:

    It looks like the service is still selected round robin. Is there a way to select the service to call? For example I may have multiple versions of a product on the fabric. Is it possible to select the one to use?


    You are correct. In my previous code example, the initial service was selected via round-robin. That service was used for all subsequent requests (by obtaining the service_id from the initial response).


    If you would like to invoke a specific instance of a service, you can use the "service registry". The topic /mcafee/service/dxl/svcregistry/query is used to query the service registry. The ability to query the registry is pretty limited at this point. If you pass an empty JSON object you will receive information on all of the services registered. If you specify a serviceType in the JSON object you will receive information on all services of that particular type. For example, you could query for all Threat Intelligence Exchange (TIE) services.


    The following code example registers 3 different services. The first two services are of the same type ("myService"), while the third service is a different type ("anotherService"). The code then performs two different queries on the service registry. The first query returns all services, while the second query returns services of the type, "myService".



    The output for the first query should appear similar to the following:



    The output for the second query should appear similar to the following:



    As you can see there is quite a bit of information on each service. It is also important to note that the service registration information includes a serviceGuid. This value can be specified in the service_id of a request message to invoke that particular service. Other useful information includes metaData which could be used as a means of selecting the service to invoke.


    Hope this helps,

    Chris

  • Multiple services?

    Service zones do not apply any restrictions. Their purpose is merely to route service requests to services that are available within the current zone. If a service is not found in that zone, it will look in a parent zone. If multiple services exist in the same zone, they will round-robin.


    You can create conversation-based clients by setting the service_id on the Request message. This will route the message to a specific service instance. The typical pattern is to send the initial request without setting the service identifier. In the initial case, the broker will select the service to invoke. After the initial invocation, the service_id as obtained in the initial Response message is used for all subsequent requests.


    I updated the service example that is included with the OpenDXL Python Client SDK to create multiple services as show below:



    The output from this sample should look similar to the following:


    Python: Multiple Service Output
    1. Service received request payload: ping
    2. Client received response payload: pong from service {1c456f55-e6d2-4b4f-95e2-9011cc609d3b}
    3. Service received request payload: ping
    4. Client received response payload: pong from service {2108df2b-cde1-4773-bbb1-7e8443f44ade}
    5. Service received request payload: ping
    6. Client received response payload: pong from service {78aea342-c740-404e-83fc-6fb2ae6ecd61}
    7. Service received request payload: ping
    8. Client received response payload: pong from service {a5538f3d-d14a-4ccf-92d5-e9e3049e2b27}
    9. Service received request payload: ping
    10. Client received response payload: pong from service {4388a0f8-4908-45e6-af02-eae461602141}


    As show above, all of the service instances are invoked in a round-robin fashion. I then updated the invoking client code portion (show below) to capture the service_id from the initial request and use it on all subsequent requests.




    Once those changes have been made, the output should look similar to the following:


    Python: Single Service Invoked Output
    1. Service received request payload: ping
    2. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    3. Service received request payload: ping
    4. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    5. Service received request payload: ping
    6. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    7. Service received request payload: ping
    8. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}
    9. Service received request payload: ping
    10. Client received response payload: pong from service {452ec35e-c31a-412e-95c8-d1ea3dde637d}



    As shown above, the same service is invoked even though multiple services are registered with the fabric.


    Hope this helps,

    Chris

  • Cannot change TIE Reputation when Certificate is used for DXL authentication

    Thanks a lot for collecting that information.


    Support for certificate-based authorization was added in 3.0.1. For it to work correctly, all brokers in the fabric must be at least on that version. In the fabric displayed, only one broker is at that version (3.1.0.595). The other brokers embedded in the TIE servers are still on a 2.x version.


    Our testing of 3.1.0.595 did include a suite of tests related to certificate based authorization. However, I will configure a broker today matching that version to confirm it is working as expected.


    Thanks,

    Chris

  • DxlException: Synchronous requests may not be invoked while handling an incoming message.

    The error is occurring due to the fact that a synchronous request is being made during the handling of an event message. The reason this is not allowed is that a synchronous request will result in a response message being sent back to the invoking client.


    The pool of handlers for incoming messages in the Python OpenDXL client is common for all message types. Therefore, deadlock could occur if all of the threads were busy handling incoming events and synchronous requests were made on them. There would be no available threads to handle the response messages (they would all be consumed by the event handling threads).


    The simple answer is to make the VirusTotal service invocation on a thread other than the event handler thread. However, this is such a common use case that it is something that is handled by the OpenDXL Bootstrap application.


    I put together a quick application using bootstrap that handles your particular use case.


    Here is the application template that I used:



    The important property to note is separateThread. This property causes the event handler to be invoked on a thread other than the incoming message thread. See the following Wiki page for additional information.


    After I generated the application using OpenDXL bootstrap I had to modify the default event handler code to incorporate your logic.


    Here is the updated eventhandlers.py file (the original file was generated by bootstrap):



    The modifications I made to the handler are as follows:

    • Changed the base class from EventCallback to ReputationChangeCallback to match your handler
    • Replaced the generated on_event method with your on_reputation_change method implementation
    • Instead of passing the client to the constructor like you were doing, I used the client that is available via the generated application (self._app.client).

    I tested the generated application and I am able to successfully perform VirusTotal lookups when reputation change events are received.


    Hope this helps,

    Chris

  • Error connecting to broker

    Yes, there are plans to simplify the way certificates are provisioned for OpenDXL clients. In ePO-managed environments the client will provide an option to automatically generate a key-pair, trigger signing via ePO, and pull down the necessary files for connecting to brokers.

  • Error connecting to broker

    That error indicates that the broker is not aware of the CA that was used to sign the certificate of the client that is attempting to connect.


    A couple of things you can try (only applicable for an ePO-managed DXL environment):

    • Confirm that you imported the CA that signed your client's certificate in the "DXL Certificates" server setting.
    • Ensure the broker has received the CA (force a wakeup, etc.).

    If you have access to the broker, you can look at the following file to see if the broker does indeed have the CA:


    /var/McAfee/dxlbroker/keystore/ca-client.crt


    If you have a basic installation (not using multi-ePO, etc.) there will be two certificates in this file by default. Once you add your CA you should see three in there. If there are still only two, that indicates that the broker has not received your CA via policy.


    Chris