Table 1.1. Zebra document model
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Complex semi-structured Documents | XML and GRS-1 Documents | Both XML and GRS-1 documents exhibit a DOM like internal representation allowing for complex indexing and display rules | Chapter 8, ALVIS XML Record Model and Filter Module and Chapter 9, GRS-1 Record Model and Filter Modules |
| Input document formats | XML, SGML, Text, ISO2709 (MARC) | A system of input filters driven by regular expressions allows most ASCII-based data formats to be easily processed. SGML, XML, ISO2709 (MARC), and raw text are also supported. | Section 2.5, “Record Models and Filter Modules” |
| Document storage | Index-only, Key storage, Document storage | Data can be, and usually is, imported into Zebra's own storage, but Zebra can also refer to external files, building and maintaining indexes of "live" collections. |
Table 1.2. Zebra search functionality
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Query languages | CQL and RPN/PQF | The type-1 Reverse Polish Notation (RPN) and its textual representation Prefix Query Format (PQF) are supported. The Common Query Language (CQL) can be configured as a mapping from CQL to RPN/PQF | Section 1.1.1, “Prefix Query Format (PQF)” and Section 4, “Server Side CQL to PQF Query Translation” |
| Complex boolean query tree | CQL and RPN/PQF | Both CQL and RPN/PQF allow atomic query parts (APT) to be combined into complex boolean query trees | Section 2.1, “RPN tree structure” |
| Field search | user defined | Atomic query parts (APT) are either general, or directed at user-specified document fields | Section 2.1.3, “Atomic queries (APT)”, Section 2.1.5, “Zebra's special access point of type 'string'”, Section 2.3.1, “Use Attributes (type 1)”, and Section 3.4.1, “IDXPATH Use Attributes (type = 1)” |
| Data normalization | user defined | Data normalization, text tokenization and character mappings can be applied during indexing and searching | Chapter 10, Field Structure and Character Sets |
| Predefined field types | user defined | Data fields can be indexed as phrase, as into word tokenized text, as numeric values, URLs, dates, and raw binary data. | Section 2, “Charmap Files” and Section 3.5.2, “Mapping of PQF APT structure and completeness to register type” |
| Regular expression matching | available | Full regular expression matching and "approximate matching" (e.g. spelling mistake corrections) are handled. | Section 3.6, “Zebra Regular Expressions in Truncation Attribute (type = 5)” |
| Term truncation | left, right, left-and-right | The truncation attribute specifies whether variations of one or more characters are allowed between search term and hit terms, or not. Using non-default truncation attributes will broaden the document hit set of a search query. | Section 2.4.4, “Truncation Attributes (type = 5)” |
| Fuzzy searches | Spelling correction | In addition, fuzzy searches are implemented, where one spelling mistake in search terms is matched | Section 2.4.4, “Truncation Attributes (type = 5)” |
Table 1.3. Zebra index scanning
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Scan | term suggestions | Scan on a given named index returns all the
indexed terms in lexicographical order near the given start
term. This can be used to create drop-down menus and search
suggestions. | Section 1.2.3, “Scan Operation” and Section 2.1.3, “Atomic queries (APT)” |
| Facetted browsing | available | Zebra 2.1 and allows retrieval of facets for a result set. | Section 3.3, “Zebra specific Scan Extensions to all Attribute Sets” |
| Drill-down or refine-search | partially | scanning in result sets can be used to implement drill-down in search clients | Section 3.3, “Zebra specific Scan Extensions to all Attribute Sets” |
Table 1.4. Zebra document presentation
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Hit count | yes | Search results include at any time the total hit count of a given query, either exact computed, or approximative, in case that the hit count exceeds a possible pre-defined hit set truncation level. | Section 3.2.4, “Local Approximative Limit Attribute (type 11)” and Section 2, “The Zebra Configuration File” |
| Paged result sets | yes | Paging of search requests and present/display request can return any successive number of records from any start position in the hit set, i.e. it is trivial to provide search results in successive pages of any size. | |
| XML document transformations | XSLT based | Record presentation can be performed in many pre-defined XML data formats, where the original XML records are on-the-fly transformed through any preconfigured XSLT transformation. It is therefore trivial to present records in short/full XML views, transforming to RSS, Dublin Core, or other XML based data formats, or transform records to XHTML snippets ready for inserting in XHTML pages. | Section 2.2, “ALVIS Exchange Formats” |
| Binary record transformations | MARC, USMARC, MARC21 and MARCXML | post-filter record transformations | |
| Record Syntaxes | Multiple record syntaxes for data retrieval: GRS-1, SUTRS, XML, ISO2709 (MARC), etc. Records can be mapped between record syntaxes and schemas on the fly. | ||
| Zebra internal metadata | yes | Zebra internal document metadata can be fetched in SUTRS and XML record syntaxes. Those are useful in client applications. | Section 4, “Retrieval of Zebra internal record data” |
| Zebra internal raw record data | yes | Zebra internal raw, binary record data can be fetched in SUTRS and XML record syntaxes, leveraging %zebra; to a binary storage system | Section 4, “Retrieval of Zebra internal record data” |
| Zebra internal record field data | yes | Zebra internal record field data can be fetched in SUTRS and XML record syntaxes. This makes very fast minimal record data displays possible. | Section 4, “Retrieval of Zebra internal record data” |
Table 1.5. Zebra sorting and ranking
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Sort | numeric, lexicographic | Sorting on the basis of alpha-numeric and numeric data is supported. Alphanumeric sorts can be configured for different data encodings and locales for European languages. | Section 9.4, “Sorting” and Section 3.2.1, “Zebra Extension Embedded Sort Attribute (type 7)” |
| Combined sorting | yes | Sorting on the basis of combined sorts e.g. combinations of ascending/descending sorts of lexicographical/numeric/date field data is supported | Section 9.4, “Sorting” |
| Relevance ranking | TF-IDF like | Relevance-ranking of free-text queries is supported using a TF-IDF like algorithm. | Section 9.3, “Dynamic Ranking” |
| Static pre-ranking | yes | Enables pre-index time ranking of documents where hit lists are ordered first by ascending static rank, then by ascending document ID. | Section 9.2, “Static Ranking” |
Table 1.6. Zebra live updates
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Incremental and batch updates | It is possible to schedule record inserts/updates/deletes in any quantity, from single individual handled records to batch updates in strikes of any size, as well as total re-indexing of all records from file system. | zebraidx(1) | |
| Remote updates | Z39.50 extended services | Updates can be performed from remote locations using the Z39.50 extended services. Access to extended services can be login-password protected. | Section 10, “Extended Services: Remote Insert, Update and Delete” and Section 2, “The Zebra Configuration File” |
| Live updates | transaction based | Data updates are transaction based and can be performed on running Zebra systems. Full searchability is preserved during life data update due to use of shadow disk areas for update operations. Multiple update transactions at the same time are lined up, to be performed one after each other. Data integrity is preserved. | Section 8, “Safe Updating - Using Shadow Registers” |
Table 1.7. Zebra networked protocols
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Fundamental operations | Z39.50/SRU explain,
search, scan, and
update | Section 1.2, “Operation types” | |
| Z39.50 protocol support | yes | Protocol facilities supported are:
init, search,
present (retrieval),
Segmentation (support for very large records),
delete, scan
(index browsing), sort,
close and support for the update
Extended Service to add or replace an existing XML
record. Piggy-backed presents are honored in the search
request. Named result sets are supported. | the section called “Z39.50 Protocol Support and Behavior” |
| Web Service support | SRU | The protocol operations explain,
searchRetrieve and scan
are supported. CQL to internal
query model RPN
conversion is supported. Extended RPN queries
for search/retrieve and scan are supported. | the section called “SRU Protocol Support and Behavior” |
Table 1.8. Zebra data size and scalability
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| No of records | 40-60 million | ||
| Data size | 100 GB of record data | Zebra based applications have successfully indexed up to 100 GB of record data | |
| Scale out | multiple discs | ||
| Performance | O(n * log N) | Zebra query speed and performance is affected roughly by
O(log N),
where N is the total database size, and by
O(n), where n is the
specific query hit set size. | |
| Average search times | Even on very large size databases hit rates of 20 queries per seconds with average query answering time of 1 second are possible, provided that the boolean queries are constructed sufficiently precise to result in hit sets of the order of 1000 to 5.000 documents. | ||
| Large databases | 64 bit file pointers | 64 file pointers assure that register files can extend the 2 GB limit. Logical files can be automatically partitioned over multiple disks, thus allowing for large databases. |
Table 1.9. Zebra supported platforms
| Feature | Availability | Notes | Reference |
|---|---|---|---|
| Linux | GNU Linux (32 and 64bit), journaling Reiser or (better) JFS file system on disks. NFS file systems are not supported. GNU/Debian Linux packages are available | Section 2, “GNU/Debian” | |
| Unix | tar-ball | Zebra is written in portable C, so it runs on most Unix-like systems. Usual tar-ball install possible on many major Unix systems | Section 1, “UNIX” |
| Windows | NT/2000/2003/XP | Zebra runs as well on Windows (NT/2000/2003/XP). Windows installer packages available | Section 3, “Windows” |