NMPDR Development Blog

The holiday is over and NMPDR version 18 is now officially loaded on the development server. In addition, the keyword search now supports three-letter words. For example, in the old system, adenine RNA and adenine both return the same result set because the three-letter word RNA is ignored.. In the new system, adenine returns 1702 records and adenine RNA returns only 44 records.

The difference report for version 18 is here.

• New Development Site loaded: January 2, 2006
• Latest code and data copied to Staging Site for beta testing: January 10, 2006
• Version 18 published to Public Site: January 12, 2006

I am testing a change to the attribute server code, so attributes on the Development Server will be unpredictable for a while. The idea is to permit full-text searching on attribute keys and values. Here is the plan.

• Update the attribute DBD to support full-text searching. This includes creating a searchable keyword field in the HasValueFor table. The keyword field will contain a cleaned copy of the key name and value.
• Update CustomAttributes.pm to support the new searchable field.
• Reload the development server's test attribute database.
• Test adding and deleting attributes.
• Test the performance of the ev_code_cron job in the new environment.
• Convert the FIG.pm text search to use the new attribute-searching method.
• Add the new attribute-searching method to the XMLRPC support module.
• Move the changes to the real attribute server.

Full-text searching has now been implemented on the Development Server. A dry run of the trna search took about 5 seconds to find the values, which is much better than a minute.

The development server, therefore, is once again fully-functional; however, additional testing is needed before this change can be ported to the attribute server itself. In particular, we need to see if the new index is going to significantly slow the ev_code_cron job.

• The new Listeria genomes (393133.3, 401650.3, 393128.3, 393117.3, 393121.3, 393123.3, 393124.3, 393125.3, 393130.3, 393131.3) have been marked as NMPDR, so they now appear on the Listeria page. This change will need to be ported to the annotator SEED.
• The Listeria and Strep search formlets had spelling errors in the form details, resulting in a 404 error. These have been fixed.
• There is now an anti-spam CAPTCHA code protecting the mailing list signup page. The mailing list has been hit hard by spammers. Unfortunately, the change required removing the signup formlets on the organism pages.

I have commented out the overflow-x and overflow-y rules in the main NMPDR CSS file. The directional overflow rules are still in the not-quite-supported category of CSS. In the Mac version of Firefox 2, when the content height became too large, the browser went berserk and created a ghastly blank page with things popping in and out of existence. This is, in defiance of all sense and logic, only a problem on the Mac. On Windows the page displayed just fine.

The page that exhibited the problem was the database design page.

Thanks to Ross and Kaitlyn for donating their laptops so I could test this problem.

In other news, there will be a delay in the schedule for implementing attributes and rolling out NMPDR version 18 because I still can't use VPN. Finding a way to work around that will be my main focus tomorrow.

The new attribute system has been ported to the attribute server and now appears to be running fairly well. A few things have changed.

• The find_attributes method, which allowed searching for substrings inside keywords and values, is no longer supported.
• Please note that the hundred-by-hundred attributes have changed. There is now a collection attribute that specifies the names of the collection in which an object participates. So, for example, to get all objects in the Higher Plants collection, you would call $fig->get_attributes(undef, 'collection', 'higher_plants').
• The erase_attribute_entirely method deletes all values for a keyword but does not erase the notion of the keyword itself. That's a good thing, because if you erase the notion in the new system you can't add values back. To add or remove a keyword, you must use the Attribute Control Panel at http://anno-3.nmpdr.org/attrib_server/Attributes.cgi.
• There are currently 3.1 million attributes in the database.

• A previous fix that allowed Internet Explorer users to press ENTER from a search form to start a search was not working properly, with the result that users would get an error message when they were linking directly to the forms. This fix has been backed out.

The festival will continue later, after I fix an emergency problem with attributes on the NMPDR.