Good day everyone.
In the process of learning your technologies, i encountered a problem with using a dxlclient.config file with FileTransferService code examples.
When trying to run basic_store_example.py, i'm catching "dxlclient.exceptions.MalformedBrokerUriException: Invalid port", what made me a bit confused , because i used that configuration file in all previous examples (event example, service example, event subscriber, event publisher, service invoker, service provider, service wrapper etc). What can be the reason of that exception?
Also tell me if you need some additional information (screenshots, code parts etc)
Not sure why that would be happening. Can you please post the entire "Traceback"?
Thanks a lot,
I've managed that issue by editing dxlclient.config, that was located in "sample" folder (but don't know, why script used that config instead of the one i've indicated). So for now, i can't repeat that traceback, script working fine.
But now i have another issue. When running service_provider_example.py (script 1) and then running basic_service_example.py (script 2) to send a file, script 1 tracking that file, but after completed transfer script 2 crushing with next traceback:Code
- Traceback (most recent call last):
- File "/home/andrii/Desktop/ForWork/dxlfiletransferservice-python-dist-0.1.1/sample/basic/basic_service_example.py", line 101, in <module>
- File "/home/andrii/.local/lib/python3.6/site-packages/dxlfiletransferclient/client.py", line 224, in send_file_request
- File "/home/andrii/.local/lib/python3.6/site-packages/dxlfiletransferclient/client.py", line 378, in send_file_from_stream_request
- File "/home/andrii/.local/lib/python3.6/site-packages/dxlfiletransferclient/client.py", line 456, in _invoke_service
- return MessageUtils.json_payload_to_dict(response)
- File "/home/andrii/.local/lib/python3.6/site-packages/dxlbootstrap/util.py", line 62, in json_payload_to_dict
- return MessageUtils.json_to_dict(MessageUtils.decode_payload(message, enc))
- File "/home/andrii/.local/lib/python3.6/site-packages/dxlbootstrap/util.py", line 38, in json_to_dict
- return json.loads(json_string.rstrip("\0"))
- File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
- return _default_decoder.decode(s)
- File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
- obj, end = self.raw_decode(s, idx=_w(s, 0).end())
- File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode
- raise JSONDecodeError("Expecting value", s, err.value) from None
- json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
In some iterations code working fine, in some - throwing exception, without changing any parameters.
Another issue appeared with Basic Send File Request Example and Basic Store Example - they both running fine, but file did not transfer from one folder to another. Documentation says, that files must be saved in directory, that configured as storageDir in service configuration file, but i don't see any direction to this config file in code (for both Basic Send File Request and Basic Store).
Thanks in advance,
Unfortunately, it is a bit difficult to debug what is occurring. However, I went through the process to install and test the service and it seems to be working as expected. The steps are listed below. Please let me know if you have any issues when performing these steps.
1.) Download the latest service release
2.) Unzip the service release
3.) Change into service directory
4.) Provision the service
dxlclient provisionconfig config <provision-service-host> service-cn
5.) Provision the samples
dxlclient provisionconfig sample <provision-service-host> sample-cn
6.) Edit the service config and set a destination directory for transferred files
vi config/dxlfiletransferservice.configCode: File Transfer Service Configuration
7.) Start the service (force into background)
python -m dxlfiletransferservice config &
8.) Run the sample (send a file, "README.html")
python sample/basic/basic_store_example.py README.htmlCode: Output
- 2019-06-03 16:20:42,820 dxlfiletransferclient.store INFO Assigning file id '70b7b036-79ec-4fff-98a7-17b4f8c135f6' for '/tmp/.workdir/70b7b036-79ec-4fff-98a7-17b4f8c135f6'
- 2019-06-03 16:20:42,821 dxlfiletransferclient.store INFO Stored file '/tmp/README.html' for id '70b7b036-79ec-4fff-98a7-17b4f8c135f6'
- Percent complete: 100%
- Response to the request for the last segment:
- "file_id": "70b7b036-79ec-4fff-98a7-17b4f8c135f6",
- "result": "store",
- "segments_received": 1
- Elapsed time (ms): 5.20992279053
9.) Confirm the file was transferred
ls -l /tmp
I transferred multiple files without experiencing any issues. Please let me know if you experience any problems when attempting to perform the above steps.
Thanks a lot,
I already have latest version of dxlfiletransferservice, so firstly i've started from step #3. All commands worked fine, i even had the same console output as yours, but folder is empty. Then i deleted dxlfiletransferservice folder and started from step #1, but in the end got the same result - no exceptions, no mistakes, but empty destination folder.
And also, each time when i run dxlfiletransfer config (step #7) it appears in my broker services list. First one of this configs i'm starting from Kitematic as docker container.
i'm working on ESXi vmware, with Linux Ubuntu (x64) operating system. Maybe, this few details will help you to understand the root of the issue.
I also tested basic_service_example.py - it's working and sending file to indicated directory. And also, basic_send_file_request.py working fine after correct provisioning, file was transfered correctly.
Thanks a lot,
The reason it is appearing multiple times in the services list, is that the service is registering each time it is started (with a unique service identifier). That occurs if the previous service was not shut down correctly. The previous instances remain until they are either invoked (by sending a request), or their TTL expires.
If you happen to send a request to a service that no longer exists, you will receive a "ServiceNotFound" exception. The broker will round-robin service invocations. So, if you just send requests, you should see all of the stale instances disappear.
So, what is your current status? Are you still experiencing issues?
Yes, issues are still existing. Both basic_store_example and basic_send_file_request_example scripts running normally without any exceptions or errors, they both send logging in console, that says about completed transmission, but file did not transmitted from one folder to another. Don't have any idea even where to look in code for solution
Thanks in advance,
So, I would suggest following my steps exactly. I am wondering if the files themselves are being stored in the Docker container itself. In my steps, I am not using Docker at all to eliminate that possible scenario.
When i made a second attempt, i've followed your instruction step by step, but nothing changed.
Files stored on my virtual machine (but not in docker container). Directories, between which i transfer files, both placed in Documents folder.
UPD: Looks like i've troubleshooted all that moments, for now. Issue was in absence of dxlfiletransfer.config files in some folders. But when i read the documentation,I got the impression that actions with the configuration files should be performed once and then i just need to indicate it's location. Thank you for help. Also, i would recommend to add some of your steps to documentation, so it would be easier for less-skilled users to deal with it. For example, Command Line Provisioning pages has no information about provisionconfig command for the sample folder etc.
Thanks for help,
Great to hear you got it working!
As far as documentation, I agree it could definitely be clearer.
Here is the page that covers provisioning the samples for the file transfer service:
Each of the samples also mentions provisioning as a prerequisite:
How do you think this could be improved to be clearer? Any input would be greatly appreciated!