NMPDR Development Blog

I've added an animated banner to the intermediate search status page. My experience when I was in industry is that if there's a moving picture involved, it's easier to believe the search is working. I have also disabled the Annotation button so it won't show up in the final result.

The difference report is now available. It has also been put in the Quick Links list on Development Blog. The main anomaly is that none of the 29 new genomes have BBHs.

I am currently working on a glitch in the subsystem tree and a method for downloading search results. (This latter I hope will help us to understand the peculiarities of the signature genes tool.)

I have added a link to the search results page that allows you to download the entire search output as a tab-delimited file. I am adding features one at a time and testing them. Suggestions are welcome, but because it is a work in progress there is no need to worry that any anomalies are cast in stone.

The annotation history links have been missing from the NMPDR Protein Page for some time. This is now fixed.

We now have the ability to download search results to a tab-delimited file. On each search results page (for example, this one), there is now a link titled Click here to download the full search results. Clicking the link will open a download-file box and then download the search results to a specified location on your workstation. The downloaded text is cleaned of links and does not include the button columns. There is one anomaly. In Firefox, the OK button in the download dialog box appears disabled, but will work properly when clicked. (Even Tobi does not know how to fix this problem, so I don't expect an answer soon.) In the meantime, this is a useful feature I have wanted for some time and is worth keeping.

• The search box on the strep page has been fixed. Previously, it gave an error message any time you tried to search.
• The Compare Regions button on the protein page now returns the correct results. Previously, it was using BBHs instead of Sims to detect the closest genes.
• The category column in the organism drug target tables was always coming up with the Suggest link. This has been fixed. In addition, the Suggest link has been turned off so that the categories show up better.
• The last of the damaged image files have been repaired.

I am still working on the problem Ross found with the NMPDR PINS display not matching the SEED display. I know which subroutine in the Chromosomal Clusters CGI is failing, but I don't yet know where or why.

• The download link on the search results page is now marked as application/octet-stream instead of text/plain. This means it is more well-behaved in the browser, but it also means the line-endings have to be set to the proper value by the search CGI script rather than by the browser. The web page uses Javascript to detect the client operating system and then passes that information to the CGI script, which puts the correct line-endings in the file.

Remaining tasks include finding the problem in the Sprout version of the Pins (I now know it has something to do with setting the colors) and adding the serotypes to the NMPDR organism group pages.

• The Pin buttons are finally working again on the NMPDR development server protein page.
• The serotypes are now permanent fixtures on the core organism group pages (e.g. Listeria). The serotype data that Leslie entered manually some time back has been stored on the attribute server. The page generator looks at all the genomes in the group, and if at least one has a known serotype, the serotype column appears to the right of the genome name.

• New Development Site loaded: March 29, 2007
• Latest code and data copied to Staging Site for beta testing: April 11, 2007
• Version 19 published to Public Site: April 13, 2007

• The Listeria innocua and Listeria welshmeri have been reclassified as belonging to the other Listeria group. In addition, their serotype information has been loaded onto the attribute server and now appears on the Listeria group page.
• All the numerous changes over the past month have been checked into CVS, including the massive renaming of DBObject to ERDBObject so that it no longer conflicts with PPO, the new Sprout methods for SEED Viewer support, the fix to Pins and Compare Regions so that they work properly in Sprout, and a bunch of other small stuff.

The Development NMPDR is now code complete for version 19 (which is due to move to the staging server on Wednesday night). This means all we have left are bug fixes and cover page improvements.

The PEGs Only check box has been removed from the signature genes tool. In almost all cases it makes no difference. When that situation changes, we can add it back.

NMPDR version 19 has now been moved to the staging server. There was a slight delay in the original schedule due to a problem Leslie discovered with the organism pages. Bob fixed it this morning, however, so now we are back on track.

I have pushed the cut-over date forward to April 17 so that there will still be two full business days for testing.

• The subsystem tree was displaying incorrectly in Internet Explorer. This has been fixed.
• It is now possible to download search results in FASTA format.
• You can now specify a favored alias type (e.g. uni, gi) for searches. The favored alias type is shown at the beginning of the alias list and is also used as the gene ID in the FASTA downloads.

These changes have been copied to the Staging Server,

I am in the midst of applying the following changes.

• The subsystem count at the bottom of the organism summary page will no longer be active as a link. Previously, this took the user into the SEED and there was no real way to get back from that. It was easier to simply disable the link in NMPDR than convert the entire subsys_vector script.
• I am hacking in temporary fixes for the download page. The FTP directory for Listeria now has the full set of organisms. I am creating ZIP files on web-1 to be used as targets for the HTTP downloads. We need some automated way to keep this data up to date, and I have a request into Bob for any ideas he might have.
• I'm adding an operon search. (I should not have done this, but I felt I had to code it before I forgot how.) This will not be open to the public, but will be made available for Ron Taylor so he can download the SEED data he wants from the NMPDR.

Once these fixes are in and I've tested them on the development server, I will copy them to the staging server, test them there, and then roll the version.

• The subsystem count at the bottom of the organism summary page is no longer be active as a link. Previously, this took the user into the SEED and there was no real way to get back from that. It was easier to simply disable the link in NMPDR than convert the entire subsys_vector script.
• The downloads page presents an FTP directory when the user clicks the FTP link. When the user clicks the HTTP link, a ZIP file of the directory is downloaded to the user's hard drive.
• There is now an Operon search. Like the drug search, it's available on an orphaned page not generally visible to the public. The user specifies a genome and some tuning parameters and is presented with the genes in the genome grouped by operon. In addition to the standard search results fields, the operon search also displays the feature location and the feature's upstream DNA. For forward operons, the genes are presented in the normal order. For backward operons, the genes are presented in the reverse order so that the upstream data for the first gene of an operon is always upstream of the entire operon. This is almost what Ron Taylor wanted, the exception being that the downloaded file only includes the subsystem names and not the links. My goal here was to prove I could create a custom search in one day. It was one-and-a-half days, but I'm getting closer every time.

All of this has been moved to the staging server, and the links herein are staging server links.

Somehow when I updated the attribute server last night a bogus .htaccess file got pulled in. This file caused an Internal Server Error each time the attribute server was accessed.

The goal of the file was to prevent unauthorized access to the Attribute Control Center. The problem was that it used a directive which works on the Annotator SEED, but not in the attribute server. so every use of a CGI script on the attribute server failed with a message about an error in the .htaccess file. The tricky part of fixing it was allowing general access to the attribute data without allowing general access to the control center. I accomplished this using a FilesMatch directive, a process which I believe will be causing me nightmares for several weeks.

Anyway, the attribute server is back, and is considerably more secure than it used to be.

There are now methods in FIGRules.pm for converting BLAST scores to and from the sortable form used in the attribute database. EncodeScore converts a floating-point score to a sortable string. DecodeScore converts the sortable string back to its floating-point value.

The new query_attributes method in FIG.pm allows you to specify an SQL-style filter string when searching for attributes. This capability is necessary for situations where we want to restrict an attribute retrieval to a particular range of values, as is required to get the best docking results for a drug target PDB.

Finally, version 20 of Sprout, which will start loading this weekend, has been converted to use the attribute server instead of its own Property tables. The Property tables will only contain attributes used to speed certain types of searches.