NMPDR Development Blog

Mark and I spent the afternoon tuning the pattern scan and the compare regions on the development NMPDR. The entry point for the pattern scan / blast search is here. From the results page, you can get to a standard protein page using the NMPDR button and Mark's context display using the Context button. We were consistently able to get the response time under 20 seconds for both DNA and protein searches.

I am currently working on the code that allows the user to decide which ID (uniprot, locus tag, etc.) should be displayed in the results. Once that's done, I will fold it into the display code for the various feature searches as part of getting them fixed and adding rollover hints to the search forms. (This is all based on a discussion with Folker and Liz last week.) In the meantime, I am running an attribute report to find out which attributes can be added to the searching. Once that's done, I will need to reload the feature table to get the new attributes, at which point the site will be mostly working except for the help pages and stuff.

Things are slowly coming together for NMPDR version 21. The word search is mostly working, and I am finishing the fixes to the other feature-based search. Once this is done, the remaining tasks are

• Adding an XML download format.
• Reinstalling the FASTA download format for feature-based searches.
• Fixing the Drug Search. This is not a search for drug targets, but rather a report on the docking results we already have.

There was a bug in the pattern search because the formatter was removing spaces from the pattern string. This has been fixed.

I am currently running the difference report. I added coupling and BBH statistics, both of which have slowed the report considerably, which is why it is still going on. Since this report is not run very frequently, I don't consider that a big problem. Once the report is done, the data will appear here.

This weekend I will re-run the attribute report, which will give me the information I need to begin implementing a drug target search (as opposed to a docking result report). I don't wan to start the attribute report until the difference report is finished.

Finally, I am meeting with Ross on Monday to discuss implementing a close-strain comparison tool. This has become a little more urgent because one of the site users requested it.

The latest difference report is here. The genome-related data is now in a separate table. The old report listed all the new and deleted features, but those lists are now so large that they make the report difficult to read. Instead, the number of new and deleted features will be added to the genome report (starting with the next one). The new report contains all the data items for the semi-annual report, which means that from now on there will be no need to write custom applications every time the report comes around.

A link to the report has been placed on the front page of the NMPDR Development Blog under the title Latest Version Database Report so that it can be found easily when it is needed.

• New Development Site loaded: June 28, 2007
• Latest code and data copied to Staging Site for beta testing: July 25, 2007
• Version 21 published to Public Site: July 27, 2007

Version 21 is now considered stable enough for testing. Among the new features that will be available in this release are:

• BlastN, DNA Scan, and Protein Scan are now available on the Tool search page (which replaces the old BLAST page).
• Search output for the Tool Search page contains more information (including the matching sequence for scans and the alignment for BLAST). Each row includes a link to a compare-region display.
• It is now possible to download search results in XML format.
• The search results use real buttons instead of fake buttons simulated using styles. The buttons are automatically converted to URLs when the results are downloaded in XML or tab-delimited format.
• You can specify the type of ID to be used when displaying feature IDs in search results. If a feature does not have an ID of the specified type, it will fall back to the FIG ID.
• Coloring is available on new-style subsystem diagrams.
• The numbers required for the semi-annual report will now automatically be generated as part of the version change process.

There was also a major upgrade to the underlying code for the search. Previously, the search was heavily biased toward searches that return features. Under the new system, the type of result is decoupled from the type of search, which makes it easier to create searches for new types of objects (such as subsystems and genomes) in the future.

The SEED Viewer has been re-activated, and can be found here. There is also a link to it in the sidebar on the front page of the Development Blog. This is the old viewer, and it is currently being used to insure that the NMPDR can support the functions needed by the SEED Viewer. The idea here is that when the new viewer is ready, we will be in a better position to couple it to the rest of the site. In the meantime, because there is no direct link to it on the website, we can test and modify it without disrupting anything on the real web site.

The NMPDR Viewer now allows downloading each of the three result trees from a metabolic reconstruction. The downloads are done direct-to-disk, like the downloads from the NMPDR search results page.

This was implemented as a response to a customer requirement for the RAST Server. The next step is to move the code over to the RAST server itself.

The context view available from the BLAST/Scan search now includes additional explanatory text at the top. The pop-up box you get from clicking on a gene now shows only the NMPDR protein page. I am awaiting a response from Toby and Daniel as to whether I can get the click to go directly to the page without the intervening popup box.

When the incoming search pattern or sequence is more than 30 characters, the context view displays a character count rather than the entire sequence, because a long sequence looks really ugly.

I am in the process of re-doing the search helps and converting them to pages in the Seed Wiki. Clicking on the new-fangled blue-question-mark-things will pop up the relevant Wiki page.

In addition to the Wiki conversion, I have 11 more change requests from Leslie and 9 requests from Folker and Liz. Given that the Wiki conversion is essential and I also have to update some of the cover pages myself due to Leslie's absence, I am seriously thinking of pushing ahead the release date by another week or two. I will decide this tomorrow morning based on how much pain I'm in when I wake up.

• The output from the BLAST search is now sorted correctly. Some genes weren't being correctly identified as NMPDR core genes, causing them to float to the bottom of the list.
• The context button on the BLAST/Scan result page has been moved after the NMPDR button.
• The old favored-alias feature has been restored. In addition, when the selected ID type is not FIG, the FIG ID is shown in the organism name column in the familiar old square brackets.
• The formlets on the search output have been changed to fake buttons. This allows users to right-click on the buttons and open the linked pages in another tab or window.

I believe this covers the NMPDR-specific issues Leslie had with version 21. The remaining issues deal with the protein page and will be addressed separately.