We're proud to annonce that we freely release the Status extension !

So enjoy and feed us back so we can improve it !

WSE Status has been enhanced to be compliant with eZPublish 4.5 Community 2011.7.

This major update contains the following :

• Support of eZFind 2.5-dev (latest one) with Apache Solr 3.1
• Add of several scripts for cleaning / synchronization of index and tables
• Wildcard research on persons

Moreover, this product is now automatically tested with Selenium scripts ( around 20 tests now and still increasing ).

Next step is implementing new features. If not not already done, help us choosing the right features.

Beta is over now

Thank's a lot for beta testers helping us to fix the bugs.

The demo will be maintained, just get an account on it to try it.

• Register on the demo instance : http://wsest.demo.wascou.org/user/register
• Try WSE - Status on the demo instance : http://wsest.demo.wascou.org/status/profile

To get this extension, you can contact us directly with the contact form.

What is the next step ?

The next step is to choose next features to implement amid this ones :

• #49: Add a list system
• #50: Add a system to shorten URL
• #73: Add a moderation system

If you have other ideas, let us know, we will add it to the list.

Let a comment to this post so we can know what is the most important.

Another delivery that is correcting several issues, notably on the direct messages feature.

The search is also working back to the normal with @person searches.

Register on the demo instance : http://wsest.demo.wascou.org/user/register

Try WSE - Status on the demo instance : http://wsest.demo.wascou.org/status/profile

Thank's for Nicolas, Damien, Lars, Carlos and Ana helping me to identify bugs.

Any feedback is welcome !

Thanks to people helping us to test WSE - Status.

We have made a first bunch of correction which are the following :

• #51 : [Search]Back button
• #52 : [Search] Seconds elapsed
• #53 : [Links] #word leads to a white page
• #55 : [Search] User search does not work
• #56 : [Status] No links to the person's profile
• #57 : [Status] Order of status
• #58 : [Mine] Layout is broken
• #59 : [Favorites] Layout is broken
• #60 : [Favorites] Status are not in the favorites
• #61 : [Direct Messages] Layout is broken
• #62 : [Direct Messages] Unable to send a message
• #64 : [Mails] Link error in the New follower mail

Another delivery is planned for Monday.

Keep it in touch.

The WSE Status extension is nearly complete and we need some help to beta test the product !

The full description of the features are in the related description page.

If you want to participate to this beta, fill the contact form indicating your email, I will create you an account.

There's only 40 seats so, don't hesitate !

This is the first topic on the subject but as it deals with the WSE Status extension for eZPublish, it can be interesting to have some feedback on this feature.

So, this post is about the Search component from the eZC / AZC stack. In the following parts, we will explain what was the need, how to install and set up SolR and how the component can be used with our configuration.

As our project mainly relies on eZPublish and eZFind, we won't detail how they work but only how they have been modified to make our extension work.

The aim

Our goal was to provide a powerful search engine indexation to index extra data we put in specific tables in eZPublish. For example, let say we have the following table :

And it is perfect because eZFind is the best search engine for eZPublish. However, after a few moments diving in the code, it appears that it was not usable directly as the its implementation was designed to only index eZ Content Object and it would not do the trick.

The best option was finally to use the instance of SolR packed in eZFind and configure it to add our own data.

The solution : a custom SolR schema

This was a bit tricky due to the documentation of SolR. There's a Wiki reachable here but important are not always documented and you have to get in the XML file to look how it works.

Here are the reference that can be helpful before going further :

• Official documentation and : http://wiki.apache.org/solr/
• eZFind : http://doc.ez.no/Extensions/eZ-Find
• IBM Tutorial : http://www.ibm.com/developerworks/java/library/j-solr1/
• Le blog d'un DSI : http://leblogdundsi.lesprost.fr/article34/moteur-de-recherche-solr-sous-windows-tomcat
• Gandbox : http://www.gandbox.fr/Blogs/Technologies-Web/Developpement-avance-avec-eZ-Find-partie-1-La-gestion-des-datatypes-entre-eZ-Find-Solr

eZFind configuration

eZFind comes with two configuration sets :

• the normal one with only one core / index : your website
• the shared one with mulitple cores / index : your website in fre-FR, eng-GB, esp-SP and so on

The normal set is located in extension/ezfind/java/solr meanwhile the multicore one is located in extension/ezfind/java/solr.multicore.

A set is made of two directories :

• conf : handle the configuration
• data : handle the binary data

You may also have other directories that will be helpful to use specfic filters or external features but SolR needs those two directories at least.

By default eZFind just use the solo configuration set, so we must enable the multi core one.

In extension/ezfind/settings/ezfind.ini :

MultiCore=enabled
DefaultCore=eng-GB
LanguagesCoresMap[eng-GB]=eng-GB
LanguagesCoresMap[fre-FR]=fre-FR
LanguagesCoresMap[nor-NO]=nor-NO
LanguagesCoresMap[example]=example

This settings allow you to map

In extension/ezfind/settings/solr.ini :

Shards[]
Shards[eng-GB]=http://localhost:8983/solr/eng-GB
Shards[fre-FR]=http://localhost:8983/solr/fre-FR
Shards[nor-NO]=http://localhost:8983/solr/nor-NO
Shards[example]=http://localhost:8983/solr/example

SolR Configuration

SolR has another cool feature called the sharding that allows you to make one query on several cores / index. It's useful in case you have several index that are hetrogeneous : it means you can ask for one term in one dictionnary and have a result for all dictionaries. In eZFind, it's used to have translated result : you searchbanana and you will get results for banana in english and banane in french.

In SolR, there are three XML files to set up to have a full configuration :

• solr.xml
• solrconfig.xml
• schema.xml

solr.xml

This file is simple, it declares cores for the SolR system (sorry about this one :) ). A core is an index. In comparison, we can say that a core is a reference, like a dictionnay. You can have several dictionnaries : English, French, Spanish, Portuguese and so on. But you can also have several application domains, like dictionnary about medecine, about computer science or whatever.

Our only modification to this file was the add of a specific core :

<core name="example" instanceDir="example" />

The attributes are defined like this :

• name : name of your index / core, will be available at http://localhost:8983/<name>
• instanceDir : directory that contains all conf for this index / core (conf and data)

Then, copy the directory extension/ezfind/java/solr to extension/ezfin/java/solr.multicore/example.

solrconfig.xml

This file has been left by default for us but you may care about the language specifications made (search English for example).

schema.xml

This is the main file which will help you to map your data with fields inside Solr. But before some explanation about the SolR concept.

SolR can have different index, we've seen that just above with the core, it's useful because you can separate the index and query different index with one query (sharding).

For one index, SolR can handle several types of data :

• Structured data : identified fields that will be required for each piece of data you want to index. For example, if you want to index homogeneous documents that is in eZPublish, you will need to provide data like the node_id, the section and so on.
• Not structured data : not identified fields that will be indexed in the index and that are not required. For example, if you want to index heterogeneous documents, you can index data about a video and data about a picture even if picture and video content does not share fields together. Those fields are named dynamic fields.
• Mixed data : you can have both identified and required fields and not identified and not required fields.

In eZFind, the configuration is set to Mixed data and all content fields are required. This is why the eZFind implementation is not so extendable. So I used another client has indicated by Paul Borgermans.

ezcSearch

In ezcSearch, the eZComponents / Apache Zeta Components search component, the schema.xml that is provided is a base for what you want to do and is also Mixed Data. ezcSearch also need to implement some interfaces that are compliant with the Persistent Object definition.

I made some testing with this client and I found the following a bit strange (or I did not properly understand it) :

• There's a hard coded field called ezcsearch_type. If you know about this one, just share on the forum.
• The unique id field in the schema must be id.

More info about this in the Fisheye repository for Apache Zeta Component Search.

I finally inherited the SolR manager class to change the index function so requests can work whith my fields and I also kicked out all the static fields I don't want from the schema.xml and set my own fields. It's much better and you're free to do what you want to do.

Conclusion

SolR is very powerful and not so accessible due to in-code documentation. Maybe the best point would have been to buy a book on it before starting.

eZFind is an out-of-the-box solution that works only for eZPublish content, which is a bit restrictive in our case. According to Paul Borgermans, the next version of eZFind will be able to take care of extra non related content fields !

ezcSearch is useful to generate all the queries sent to SolR but it's too much restrictive for an out-of-the-box use. However it exists so thank you guys for having already done the job !