Welcome, Guest. Please Login or Register
UGENE Bulletin Board
  Welcome to our forum.
  HomeHelpSearchLoginRegister  
 
 
Pages: 1 2 3 4 5 
Newbie: Facing problem on CMD Tools (Read 78262 times)
Reply #30 - Feb 14th, 2012 at 10:32am

kenkl   Offline
Junior Member

Posts: 73
**
 
Hi, this is what had happened: I'm testing on the integration of ttuner into the UGENE workflow designer:

The workflow are as the first screenshot:
Read abi sequence > ttuner1 > ttuner2 > Write sequence

ttuner1: ttuner -sa $outfile -if $infile
$infile : Sequence (GFF)
$outfile: Sequence (FASTA)

ttuner2: ttuner -qa $outfile -if $infile
$infile : Sequence (GFF)
$outfile : Sequence (FASTA)

This works well when I wrote hard code with the path to the input files, but when I change it to URL input it does not work.

Kindly advice, thanks!

Regards,
Kenneth
 

ttuner1.png (127 KB | )
ttuner1.png
ttuner2.png (63 KB | )
ttuner2.png
IP Logged
 
Reply #31 - Feb 14th, 2012 at 10:59am

kenkl   Offline
Junior Member

Posts: 73
**
 
Besides, there are also some errors I mentioned earlier and hereby I attach the screenshots for your reference:
 

ttuner_test1.png (172 KB | )
ttuner_test1.png
IP Logged
 
Reply #32 - Feb 14th, 2012 at 12:25pm

German   Offline
Full Member

Posts: 118
***
 
I can comment your problems.

1) First of all, I should tell about the second schema. As I can see, your execution string is "ttuner -sa $seq -id $in". The ttuner cmdline key "-id" means that you put a directory to the input. But the "Read sequence" element reads files, and only sequence data will be available in the output port, but not any directories.

2) I have finally understood why your and PEH's attempts for using parameters doesn't work Smiley We call different things by one name (parameter). As I understand, you talk about cmdline parameters ($out, $in, ...). But I talk about Workflow element's parameters (see a screenshot). I will show how to use them.

And what about your first schema. I have already told about using ABI files in Workflow.
a) So far Workflow Designer doesn't support working with chromatogram data. When "Read Sequence" element reads an ABI file, containing both sequence and chromatogram, only sequence data will available in the output port.
b) So, when you set up a cmdline element's port you can choose only 4 formats for this port (Fasta, Fastq, GFF, Genbank). And if you want to set up your custom element for getting a sequence data from an input port, you should correctly set the execution string too (for example, be sure that $in is able to be a file with the port specified format).

I know that it is not so simple and clear, but your task is complex too (you want to unite together the work of two independent applications)

3) How can I help you? Please, look at my post
http://ugene.unipro.ru/forum/YaBB.pl?num=1326766085/0#6
You can create a custom element that would have no input ports and have output sequence port $out_seq (with the fasta format). Also, it would have a URL parameter $fof (Workflow element parameter  Wink). fof would store a url for a file of files. And the execution string would be: "ttuner -if $fof -sa $out_seq".

This element gets a file of files from the parameter, launches ttuner (that creates an output fasta file) and puts sequences to the output port.
 

parameter.JPG (33 KB | )
parameter.JPG
IP Logged
 
Reply #33 - Feb 14th, 2012 at 12:33pm

German   Offline
Full Member

Posts: 118
***
 
We are working on improving this unhandy thing. When these two issues will be resolved:
https://ugene.unipro.ru/tracker/browse/UGENE-817
https://ugene.unipro.ru/tracker/browse/UGENE-819
then you would create more handy and flexible schemes.
 
IP Logged
 
Reply #34 - Feb 14th, 2012 at 1:26pm

kenkl   Offline
Junior Member

Posts: 73
**
 
Hi German, thanks for your clarification but unfortunately it still does not work. Please see screen shots.

Besides, will future release of UGENE WD support chromatogram data? Kindly advice, thanks!
 

test2.png (187 KB | )
test2.png
test3.png (194 KB | )
test3.png
test4.png (187 KB | )
test4.png
IP Logged
 
Reply #35 - Feb 14th, 2012 at 2:18pm

German   Offline
Full Member

Posts: 118
***
 
Are you sure that your $www parameter contains exactly a url to a file of files? As I can see from screenshots, your $www is "/home/kennethtan/Desktop/ABI/forward/ITS_GA00016.1.a1.ab1". And I think that it is just an ABI file Smiley

You should create a text file "name.txt" with the next content:
/home/kennethtan/Desktop/ABI/forward/ITS_GA00016.1.a1.ab1




And some other files you need.

Also, you always could check you custom element. Just run your execution string from command line (replacing your $parameters by real files) and see the result.
Run your string:
ttuner -if /home/kennethtan/Desktop/ABI/forward/ITS_GA00016.1.a1.ab1 -sa ken.fa
and you would see that it doesn't work.

Please, try it and tell me about the results. Good luck!
 
IP Logged
 
Reply #36 - Feb 14th, 2012 at 2:20pm

German   Offline
Full Member

Posts: 118
***
 
Kenneth, see the attachment, please. There is an example of the input file, that you need to put to your $www parameter.

Check, that I wrote the correct url Smiley
 

in.txt (0 KB | )
IP Logged
 
Reply #37 - Feb 14th, 2012 at 3:18pm

kenkl   Offline
Junior Member

Posts: 73
**
 
Hi German, thanks for pointing out, it works well now! Smiley Anyway, will future release of UGENE WD support chromatogram data such as abi files? Please let me know asap, thanks!
 
IP Logged
 
Reply #38 - Feb 14th, 2012 at 4:09pm

German   Offline
Full Member

Posts: 118
***
 
kenkl wrote on Feb 14th, 2012 at 3:18pm:
Hi German, thanks for pointing out, it works well now! Smiley Anyway, will future release of UGENE WD support chromatogram data such as abi files? Please let me know asap, thanks!


Good job! Smiley
About chromatograms: currently UGENE has no any functionality for working with chromatograms, you can just open an ABI file and see the content of the file. So, Workflow Designer has no elements for working with chromatograms too.

In next releases UGENE will not support chromatogram datatype in Workflow.

But we have planned this feature for some future releases:
https://ugene.unipro.ru/tracker/browse/UGENE-838

We will be appreciate if you tell us what functionality is needed by biologist for working with chromatograms and that we could implement in UGENE. It is better to write at "Feature Requests" forum:
http://ugene.unipro.ru/forum/YaBB.pl?board=feature
 
IP Logged
 
Reply #39 - Feb 15th, 2012 at 4:30pm

kenkl   Offline
Junior Member

Posts: 73
**
 
Hi! Chromatogram data processing is important for scientific research, especially in DNA sequencing. When a genome is sequence with a automated sequencing machine the raw data are produced in abi files. Thus it would be very useful for if biologist could convert the data to a more useful format such as FASTA. When it is converted then the sequence can be used for many downstream processing. Great to hear that you guys are gonna include support for abi in future release, but could it be as soon as 1.10.3 or 1.11?

Thanks!
 
IP Logged
 
Reply #40 - Feb 16th, 2012 at 10:17am

German   Offline
Full Member

Posts: 118
***
 
kenkl wrote on Feb 15th, 2012 at 4:30pm:
Hi! Chromatogram data processing is important for scientific research, especially in DNA sequencing. When a genome is sequence with a automated sequencing machine the raw data are produced in abi files. Thus it would be very useful for if biologist could convert the data to a more useful format such as FASTA. When it is converted then the sequence can be used for many downstream processing. Great to hear that you guys are gonna include support for abi in future release, but could it be as soon as 1.10.3 or 1.11?

Thanks!

You are able to convert ABI to FASTA (or other formats) now, without any additional supporting and tools. Just make a scheme in Workflow Designer:

Read Sequence -> Write Fasta

Put your ABI files to the "Input file(s)" parameter of the "Read sequence" element. And set the output filename for the "Write Fasta" element. And you will receive the data in the needed useful format.
 
IP Logged
 
Reply #41 - Feb 16th, 2012 at 10:43am

kenkl   Offline
Junior Member

Posts: 73
**
 
Hi German, abi files consist 2 types of data: sequence and quality score. Both data are equally important in downstream processing as some bioinformatics tools require these 2 types of files, namely seq file and qul file, to process the data.

Besides I would want to ask isit possible to input 2 data from 2 different elements upstream into a single CMDline element without increasing the number of input port?

Thanks!  Smiley
 
IP Logged
 
Reply #42 - Feb 16th, 2012 at 12:10pm

German   Offline
Full Member

Posts: 118
***
 
kenkl wrote on Feb 16th, 2012 at 10:43am:
Hi German, abi files consist 2 types of data: sequence and quality score. Both data are equally important in downstream processing as some bioinformatics tools require these 2 types of files, namely seq file and qul file, to process the data.

Besides I would want to ask isit possible to input 2 data from 2 different elements upstream into a single CMDline element without increasing the number of input port?

Thanks!  Smiley

For keeping quality scores you can choose the FASTQ format.
Read Sequence -> Write Sequence
And choose "fastq" in the format property of the "Write Sequence" element.

What about the second question... Currently, it is not possible. But could you describe me a purpose of it? What different 2 input data and what external tool do you want to use? It is important for me to know about it for clear understanding the problem and finding the best solution.
 
IP Logged
 
Reply #43 - Feb 16th, 2012 at 1:16pm

kenkl   Offline
Junior Member

Posts: 73
**
 
Ok, here are the details:

I'm trying to construct a trace processing pipeline using UGENE WD. ttuner is used to call bases as well as giving quality score for each input chromatogram data. Then will use LUCY to trim off vector sequences.

Lucy is a vector sequence trimming tool and it requires at least 4 files to work:
-Vector sequence
-Splice site
-Input quality score
-Input FASTA sequence

ttuner does not output a single FASTQ file, instead it will output 2 separate files containing the quality score and FASTA sequence. Thus thus these 2 file can serve as an input data for LUCY for vector sequence removal.

Since ttuner output 2 separate files, thus I had to create 2 separate CMDline tool element for that purpose, which has been done pretty nicely. The problem arises as I could not pass 2 input files to LUCY element (created using CMDline tool), thus came my questions of whether it is possible to have 2 input data from 2 upstream elements with only 1 input port.

You can have a look at LUCY commandline through this link:
http://www.complex.iastate.edu/download/Lucy2/gadgets/lucy.pdf

Thanks for your help.

Kenneth
 

2input.png (218 KB | )
2input.png
IP Logged
 
Reply #44 - Feb 16th, 2012 at 3:12pm

German   Offline
Full Member

Posts: 118
***
 
Kenneth, thanks for your feedback.
It is very good that you tell us about needed features, because they are useful and we should implement them.

Yes, your scheme shows that several slots supporting is needed for some cmdline tools. I have created an issue at our bug tracker:
https://ugene.unipro.ru/tracker/browse/UGENE-843

It will be fixed asap.
 
IP Logged
 
Pages: 1 2 3 4 5