CustomAttributesTable of Contents

Custom SEED Attribute Manager Introduction Object IDs Retrieval and Logging FIG_Config Parameters Public Methods ArchiveFileName BackupAllAttributes BackupKeys ControlForm DeleteAttributeKey FieldMenu GetAttributeData GetGroups LoadAttributesFrom LogOperation new NewName RestoreKeys StoreAttributeKey Internal Utility Methods _KeywordString _QueryResults FIG Method Replacements AddAttribute ChangeAttribute DeleteAttribute DeleteMatchingAttributes EraseAttribute GetAttributeKeys GetAttributes QueryAttributes Key and ID Manipulation Methods AttributeTable FormID GetTargetObject JoinKey ParseID SplitKey Custom SEED Attribute Manager IntroductionThe Custom SEED Attributes Manager allows the user to upload and retriev custom data for SEED objects. It uses the ERDB database system t store the attributes.

Attributes are organized by attribute key. Attribute values ar assigned to objects. In the real world, objects have types and IDs however, to the attribute database only the ID matters. This will creat a problem if we have a single ID that applies to two objects of differen types, but it is more consistent with the original attribute implementatio in the SEED (which this implementation replaces).

The actual attribute values are stored as a relationship between the attribut keys and the objects. There can be multiple values for a single key/object pair.

Object IDsThe object ID is normally represented as

where type is the object type (Role, Coupling, etc.) and id i the actual object ID. Note that the object type must consist of only upper- an lower-case letters! Thus, GenomeGroup is a valid object type, bu genome_group is not. Given that restriction, the object ID

would represent the FIG family aclame|cluster10. For historical reasons there are three exceptions: subsystems, genomes, and features do not nee a type. So, for PEG 3361 of Streptomyces coelicolor A3(2), you simply code

The methods ParseID and FormID can be used to make this all see more consistent. Given any object ID string, ParseID will convert it to a object type and ID, and given any object type and ID, FormID wil convert it to an object ID string. The attribute database is prett freewheeling about what it will allow for an ID; however, for bes results, the type should match an entity type from a Sprout genetic database. If this rule is followed, then the database objec corresponding to an ID in the attribute database could be retrieved usin GetTargetObject method.

my $object = CustomAttributes::GetTargetObject($sprout, $idValue);Retrieval and LoggingThe full suite of ERDB retrieval capabilities is provided. In addition custom methods are provided specific to this application. To get al the values of the attribute essential in a specified Feature, yo would code

my @values = $attrDB->GetAttributes($fid, 'essential');where $fid contains the ID of the desired feature.

Keys can be split into two pieces using the splitter value defined in th constructor (the default is ::). The first piece of the key is calle the real key. This portion of the key must be defined using th web interface (Attributes.cgi). The second portion of the key is calle the sub key, and can take any value.

Major attribute activity is recorded in a log (attributes.log) in th $FIG_Config::var directory. The log reports the user name, time, an the details of the operation. The user name will almost always be unknown the exception being when it is specified in this object's constructo (see new).

FIG_Config ParametersThe following configuration parameters are used to manage custom attributes.

attrDbmsType of database manager used: mysql for MySQL or pg for PostGres.

attrDbNameName of the attribute database.

attrHostName of the host server for the database. If omitted, the current hos is used.

attrUserUser name for logging in to the database.

attrPassPassword for logging in to the database.

attrPortTCP/IP port for accessing the database.

attrSockSocket name used to access the database. If omitted, the default socke will be used.

attrDBDFully-qualified file name for the database definition XML file. This fil functions as data to the attribute management process, so if the data i moved, this file must go with it.

Public Methods new my $attrDB = CustomAttributes->new(%options);

Construct a new CustomAttributes object. The following options ar supported.

splitterValue to be used to split attribute keys into sections in th Fig Replacement Methods. The default is a double colon :: and should only be overridden in extreme circumstances.

userName of the current user. This will appear in the attribute log.

noOpenIf TRUE, the database will not be opened, but the metadata structures wil be loaded.

StoreAttributeKey$attrDB->StoreAttributeKey($attributeName, $type, $notes, \@groups);

Create or update an attribute for the database.

attributeNameName of the attribute (the real key). If it does not exist already, it will be created.

typeData type of the attribute. This must be a valid ERDB data type name.

notesDescriptive notes about the attribute. It is presumed to be raw text, not HTML.

groupsReference to a list of the groups to which the attribute should be associated This will replace any groups to which the attribute is currently attached.

DeleteAttributeKeymy $stats = $attrDB->DeleteAttributeKey($attributeName);

Delete an attribute from the custom attributes database.

attributeNameName of the attribute to delete.

RETURNReturns a statistics object describing the effects of the deletion.

NewNamemy $text = CustomAttributes::NewName();

Return the string used to indicate the user wants to add a new attribute.

ControlFormmy $formHtml = $attrDB->ControlForm($cgi, $name, \%keys);

Return a form that can be used to control the creation and modification o attributes. Only a subset of the attribute keys will be displayed, a determined by the incoming list.

cgiCGI query object used to create HTML.

nameName to give to the form. This should be unique for the web page.

keysReference to a hash mapping attribute keys to n-tuples. Each tuple will contain th attribute's data type, its description, and a list of the groups in which it participates.

RETURNReturns the HTML for a form that can be used to submit instructions to the Attributes.cgi scrip for loading, creating, displaying, changing, or deleting an attribute. Note that only the for controls are generated. The form tags are left to the caller.

LoadAttributesFrommy $stats = $attrDB->LoadAttributesFrom($fileName, %options);

Load attributes from the specified tab-delimited file. Each line of the file mus contain an object ID in the first column, an attribute key name in the secon column, and attribute values in the remaining columns. The attribute values wil be assembled into a single value using the splitter code. In addition, the key names ma contain a splitter. If this is the case, the portion of the key after the splitter i treated as a subkey.

fileNameName of the file from which to load the attributes, or an open handle for the file (This last enables the method to be used in conjunction with the CGI form uploa control.)

optionsHash of options for modifying the load process.

RETURNReturns a statistics object describing the load.

Permissible option values are as follows.

appendIf TRUE, then the attributes will be appended to existing data; otherwise, th first time a key name is encountered, it will be erased.

archiveIf specified, the name of a file into which the incoming data should be saved If resume is also specified, only the lines actually loaded will be pu into this file.

objectTypeIf specified, the specified object type will be prefixed to each object ID.

resumeIf specified, key-value pairs already in the database will not be reinserted Specify a number to start checking after the specified number of lines an then admit everything after the first line not yet loaded. Specify carefulnone to ignore this option. The defaul is none. So, if you believe that a previous load failed somewhere after 5000 lines, a resume value of 50000 would skip 50000 lines in the file, the check each line after that until it finds one not already in the database. Th first such line found and all lines after that will be loaded. On the othe hand, if you have a file of 100000 records, and some have been loaded and som not, you would use the word careful, so that every line would be checked befor it is inserted. A resume of 0 will start checking the first line of th input file and then begin loading once it finds a line not in the database.

chunkSizeNumber of lines to load in each burst. The default is 10,000.

BackupKeysmy $stats = $attrDB->BackupKeys($fileName, %options);

Backup the attribute key information from the attribute database.

fileNameName of the output file.

optionsOptions for modifying the backup process.

RETURNReturns a statistics object for the backup.

Currently there are no options. The backup is straight to a text file i tab-delimited format. Each key is backup up to two lines. The first lin is all of the data from the AttributeKey table. The second is tab-delimited list of all the groups.

RestoreKeysmy $stats = $attrDB->RestoreKeys($fileName, %options);

Restore the attribute keys and groups from a backup file.

fileNameName of the file containing the backed-up keys. Each key has a pair of lines one containing the key data and one listing its groups.

optionsHash of options affecting the load. Currently there are none.

ArchiveFileNamemy $fileName = $ca->ArchiveFileName();

Compute a file name for archiving attribute input data. The file will be in the attribute log directory

BackupAllAttributesmy $stats = $attrDB->BackupAllAttributes($fileName, %options);

Backup all of the attributes to a file. The attributes will be stored in tab-delimited file suitable for reloading via LoadAttributesFrom.

fileNameName of the file to which the attribute data should be backed up.

optionsHash of options for the backup.

RETURNReturns a statistics object describing the backup.

Currently there are no options defined.

FieldMenumy $menuHtml = $attrDB->FieldMenu($cgi, $height, $name, $keys, %options);

Return the HTML for a menu to select an attribute field. The menu wil be a standard SELECT/OPTION thing which is called "popup menu" in th CGI package, but actually looks like a list. The list will contai one selectable row per field.

cgiCGI query object used to generate HTML.

heightNumber of lines to display in the list.

nameName to give to the menu. This is the name under which the value wil appear when the form is submitted.

keysReference to a hash mapping each attribute key name to a list reference the list itself consisting of the attribute data type, its description and a list of its groups.

optionsHash containing options that modify the generation of the menu.

RETURNReturns the HTML to create a form field that can be used to select a attribute from the custom attributes system.

The permissible options are as follows.

newIf TRUE, then extra rows will be provided to allow the user to selec a new attribute. In other words, the user can select an existin attribute, or can choose a (new) marker to indicate a field t be created in the parent entity.

notesIf specified, the name of a variable for displaying the notes attache to the field. This must be in Javascript form ready for assignment So, for example, if you have a variable called notes tha represents a paragraph element, you should code notes.innerHTML If it actually represents a form field you should code notes.value If an innerHTML coding is used, the text will be HTML-escaped befor it is copied in. Specifying this parameter generates Javascript fo displaying the field description when a field is selected.

typeIf specified, the name of a variable for displaying the field' data type. Data types are a much more controlled vocabulary tha notes, so there is no worry about HTML translation. Instead, th raw value is put into the specified variable. Otherwise, the sam rules apply to this value that apply to $noteControl.

groupsIf specified, the name of a multiple-selection list control (also calle a popup menu) which shall be used to display the selected groups.

GetGroupsmy @groups = $attrDB->GetGroups();

Return a list of the available groups.

GetAttributeDatamy %keys = $attrDB->GetAttributeData($type, @list);

Return attribute data for the selected attributes. The attribut data is a hash mapping each attribute key name to a n-tuple containing th data type, the description, and the groups. This is the same format expected i the FieldMenu and ControlForm methods for the list of attributes to display.

typeType of attribute criterion: name for attributes whose names begin with th specified string, or group for attributes in the specified group.

listList containing the names of the groups or keys for the desired attributes.

RETURNReturns a hash mapping each attribute key name to its data type, description, an parent groups.

LogOperation$ca->LogOperation($action, $target, $description);

Write an operation description to the attribute activity log ($FIG_Config::var/attributes.log).

actionAction being logged (e.g. Delete Group or Load Key).

targetID of the key or group affected.

descriptionShort description of the action.

Internal Utility Methods _KeywordStringmy $keywordString = $ca->_KeywordString($key, $value);

Compute the keyword string for a specified key/value pair. This consists of th key name and value converted to lower case with underscores translated to spaces.

This method is for internal use only. It is called whenever we need to update o insert a HasValueFor record.

keyName of the relevant attribute key.

targetID of the target object to which this key/value pair will be associated.

valueThe value to store for this key/object combination.

RETURNReturns the value that should be stored as the keyword string for the specifie key/value pair.

_QueryResultsmy @attributeList = $attrDB->_QueryResults($query, @values);

Match the results of a HasValueFor query against value criteria and retur the results. This is an internal method that splits the values coming bac and matches the sections against the specified section patterns. It serve as the back end to GetAttributes and FindAttributes.

queryA query object that will return the desired HasValueFor records.

valuesList of the desired attribute values, section by section. If undef%) at the end In that case, all values that match up to and not including the percent sig will match. You may also specify a regular expression enclose in slashes. All values that match the regular expression will be returned. Fo performance reasons, only values have this extra capability.

RETURNReturns a list of tuples. The first element in the tuple is an object ID, th second is an attribute key, and the remaining elements are the sections o the attribute value. All of the tuples will match the criteria set forth i the parameter list.

FIG Method ReplacementsThe following methods are used by FIG.pm to replace the previous attribute functionality Some of the old functionality is no longer present: controlled vocabulary is no longe supported and there is no longer any searching by URL. Fortunately, neither of thes capabilities were used in the old system.

The methods here are the only ones supported by the RemoteCustomAttributes object The idea is that these methods represent attribute manipulation allowed by all users, whil the others are only for privileged users with access to the attribute server.

In the previous implementation, an attribute had a value and a URL. In this implementation each attribute has only a value. These methods will treat the value as a list with the individua elements separated by the value of the splitter parameter on the constructor (new). The defaul is double colons ::.

So, for example, an old-style keyword with a value of essential and a URL o http://www.sciencemag.org/cgi/content/abstract/293/5538/2266 using the defaul splitter value would be stored as

http://www.sciencemag.org/cgi/content/abstract/293/5538/2266The best performance is achieved by searching for a particular key for a specifie feature or genome.

GetAttributesmy @attributeList = $attrDB->GetAttributes($objectID, $key, @values);

In the database, attribute values are sectioned into pieces using a splitte value specified in the constructor (new). This is not a requirement o the attribute system as a whole, merely a convenience for the purpose o these methods. If a value has multiple sections, each sectio is matched against the corresponding criterion in the @valuePatterns list.

This method returns a series of tuples that match the specified criteria. Each tupl will contain an object ID, a key, and one or more values. The parameters to thi method therefore correspond structurally to the values expected in each tuple. I addition, you can ask for a generic search by suffixing a percent sign (%) to an of the parameters. So, for example,

my @attributeList = $attrDB->GetAttributes('fig|100226.1.peg.1004', 'structure%', 1, 2);would return something like

Use of undef in any position acts as a wild card (all values). You can also specif a list reference in the ID column. Thus,

my @attributeList = $attrDB->GetAttributes(['100226.1', 'fig|100226.1.%'], 'PUBMED');would get the PUBMED attribute data for Streptomyces coelicolor A3(2) and all it features.

In addition to values in multiple sections, a single attribute key can have multipl values, so even

my @attributeList = $attrDB->GetAttributes($peg, 'virulent');which has no wildcard in the key or the object ID, may return multiple tuples.

Value matching in this system works very poorly, because of the way multiple values ar stored. For the object ID, key name, and first value, we create queries that filter for th desired results. On any filtering by value, we must do a comparison after the attributes ar retrieved from the database, since the database has no notion of the multiple values, whic are stored in a single string. As a result, queries in which filter only on value end u reading a lot more than they need to.

objectIDID of object whose attributes are desired. If the attributes are desired for multipl objects, this parameter can be specified as a list reference. If the attributes ar desired for all objects, specify undef or an empty string. Finally, you can specif attributes for a range of object IDs by putting a percent sign (%) at the end.

keyAttribute key name. A value of undef or an empty string will match al attribute keys. If the values are desired for multiple keys, this parameter can b specified as a list reference. Finally, you can specify attributes for a range o keys by putting a percent sign (%) at the end.

AddAttribute$attrDB->AddAttribute($objectID, $key, @values);

Add an attribute key/value pair to an object. This method cannot add a new key, merel add a value to an existing key. Use StoreAttributeKey to create a new key.

objectIDID of the object to which the attribute is to be added.

keyAttribute key name.

valuesOne or more values to be associated with the key. The values are joined together wit the splitter value before being stored as field values. This enables GetAttributes::.

DeleteAttribute$attrDB->DeleteAttribute($objectID, $key, @values);

Delete the specified attribute key/value combination from the database.

objectIDID of the object whose attribute is to be deleted.

keyAttribute key name.

valuesOne or more values associated with the key. If no values are specified, then all value will be deleted. Otherwise, only a matching value will be deleted.

DeleteMatchingAttributesmy @deleted = $attrDB->DeleteMatchingAttributes($objectID, $key, @values);

Delete all attributes that match the specified criteria. This is equivalent t calling GetAttributes and then invoking DeleteAttribute for eac row found.

objectIDID of object whose attributes are to be deleted. If the attributes for multipl objects are to be deleted, this parameter can be specified as a list reference. I attributes are to be deleted for all objects, specify undef or an empty string Finally, you can delete attributes for a range of object IDs by putting a percen sign (%) at the end.

keyAttribute key name. A value of undef or an empty string will match al attribute keys. If the values are to be deletedfor multiple keys, this parameter can b specified as a list reference. Finally, you can delete attributes for a range o keys by putting a percent sign (%) at the end.

valuesList of the desired attribute values, section by section. If undef%) at the end In that case, all values that match up to and not including the percent sig will match. You may also specify a regular expression enclose in slashes. All values that match the regular expression will be deleted. Fo performance reasons, only values have this extra capability.

RETURNReturns a list of tuples for the attributes that were deleted, in th same form as GetAttributes.

ChangeAttribute$attrDB->ChangeAttribute($objectID, $key, \@oldValues, \@newValues);

Change the value of an attribute key/value pair for an object.

objectIDID of the genome or feature to which the attribute is to be changed. In general, an ID tha starts with fig| is treated as a feature ID, and an ID that is all digits and period is treated as a genome ID. For IDs of other types, this parameter should be a referenc to a 2-tuple consisting of the entity type name followed by the object ID.

keyAttribute key name. This corresponds to the name of a field in the database.

oldValuesOne or more values identifying the key/value pair to change.

newValuesOne or more values to be put in place of the old values.

EraseAttribute$attrDB->EraseAttribute($key);

Erase all values for the specified attribute key. This does not remove th key from the database; it merely removes all the values.

keyKey to erase. This must be a real key; that is, it cannot have a subke component.

GetAttributeKeysmy @keyList = $attrDB->GetAttributeKeys($groupName);

Return a list of the attribute keys for a particular group.

groupNameName of the group whose keys are desired.

RETURNReturns a list of the attribute keys for the specified group.

QueryAttributesmy @attributeData = $ca->QueryAttributes($filter, $filterParms);

Return the attribute data based on an SQL filter clause. In the filter clause the name $object should be used for the object ID, $key should be used fo the key name, $subkey for the subkey value, and $value for the value field.

filterFilter clause in the standard ERDB format, except that the field names are $object fo the object ID field, $key for the key name field, $subkey for the subkey field and $value for the value field. This abstraction enables us to hide the details o the database construction from the user.

filterParmsParameters for the filter clause.

RETURNReturns a list of tuples. Each tuple consists of an object ID, a key (with optional subkey), an one or more attribute values.

Key and ID Manipulation Methods ParseIDmy ($type, $id) = CustomAttributes::ParseID($idValue);

Determine the type and object ID corresponding to an ID value from the attribute database Most ID values consist of a type name and an ID, separated by a colon (e.g. Family:aclame|cluster10) however, Genomes, Features, and Subsystems are not stored with a type name, so we need t deduce the type from the ID value structure.

The theory here is that you can plug the ID and type directly into a Sprout database method, a follows

my ($type, $id) = CustomAttributes::ParseID($attrList[$num]->[0]);my $target = $sprout->GetEntity($type, $id);idValueID value taken from the attribute database.

RETURNReturns a two-element list. The first element is the type of object indicated by the ID value and the second element is the actual object ID.

FormIDmy $idValue = CustomAttributes::FormID($type, $id);

Convert an object type and ID pair into an object ID string for the attribute system. Subsystems genomes, and features are stored in the database without type information, but all other object ID must be prefixed with the object type.

typeRelevant object type.

idID of the object in question.

RETURNReturns a string that will be recognized as an object ID in the attribute database.

GetTargetObjectmy $object = CustomAttributes::GetTargetObject($erdb, $idValue);

Return the database object corresponding to the specified attribute object ID. Th object type associated with the ID value must correspond to an entity name in th specified database.

erdbERDB object for accessing the target database.

idValueID value retrieved from the attribute database.

RETURNReturns a ERDBObject for the attribute value's target object.

SplitKeymy ($realKey, $subKey) = $ca->SplitKey($key);

Split an external key (that is, one passed in by a caller) into the real key and the sub key The real and sub keys are separated by a splitter value (usually ::). If there is no splitter then the sub key is presumed to be an empty string.

keyIncoming key to be split.

RETURNReturns a two-element list, the first element of which is the real key and the second element o which is the sub key.

JoinKeymy $key = $ca->JoinKey($realKey, $subKey);

Join a real key and a subkey together to make an external key. The external key is the attribute ke used by the caller. The real key and the subkey are how the keys are represented in the database. Th real key is the key to the AttributeKey entity. The subkey is an attribute of the HasValueForrealKeyThe real attribute key.

subKeyThe subordinate portion of the attribute key.

RETURNReturns a single string representing both keys.

AttributeTablemy $tableHtml = CustomAttributes::AttributeTable($cgi, @attrList);

Format the attribute data into an HTML table.

cgiCGI query object used to generate the HTML

attrListList of attribute results, in the format returned by the GetAttributes o QueryAttributes methods.

RETURNReturns an HTML table displaying the attribute keys and values.