Sunday, September 20, 2015

Installing couchdb-lucene on MacOS X

Install couchdb-lucene

The original documentation can be found at https://github.com/rnewson/couchdb-lucene. I basically followed this documentation but there are a few modifications I wanted to note/remember.

I've installed couchdb-lucene with Homebrew. You can obtain Homebrew from http://brew.sh/. I've actually installed CouchDb through Homebrew too.

https://github.com/rnewson/couchdb-lucene

Configure CouchDB local.ini file

I've configured /usr/local/etc/couchdb/local.ini with following settings below.

[couchdb]
os_process_timeout=60000 ;increase the timeout from 5 seconds.

[external]
fti=/usr/bin/python /usr/local/Cellar/couchdb-lucene/1.0.2/libexec/couchdb-external-hook.py

[httpd_db_handlers]
_fti = {couch_httpd_external, handle_external_req, <<"fti">>}

Start couchdb-lucene

To start couchdb-lucene temporary in the console window run following command:

/usr/local/opt/couchdb-lucene/bin/cl_run

To start couchdb-lucene in the background run following command:

/usr/local/opt/couchdb-lucene/bin/cl_run &

If you get a connection refused error it most likely means that couchdb-lucene is not running.

Restart CouchDB

Next you need to restart couchdb.

sudo launchctl stop org.apache.couchdb
sudo launchctl start org.apache.couchdb

Once you've done that you should see following output when running "ps ax" to display the processes. You should see the last line with the couchdb-external-hook.py. If you don't see this line it means that something is wrong with your local.ini configuration.

charms:bin$ ps ax | grep couchdb
  543   ??  S      2:31.28 /usr/local/Cellar/erlang/17.4/lib/erlang/erts-6.3/bin/beam.smp -Bd -K true -A 4 -- -root /usr/local/Cellar/erlang/17.4/lib/erlang -progname erl -- -home /Users/charms -- -noshell -noinput -os_mon start_memsup false start_cpu_sup false disk_space_check_interval 1 disk_almost_full_threshold 1 -sasl errlog_type error -couch_ini /usr/local/etc/couchdb/default.ini /usr/local/etc/couchdb/local.ini /usr/local/etc/couchdb/local.d/couchdb-lucene.ini -s couch
 1677   ??  Ss     0:00.06 /usr/local/Cellar/couchdb/1.6.1_1/bin/couchjs /usr/local/Cellar/couchdb/1.6.1_1/share/couchdb/server/main.js
 1898   ??  Ss     0:00.14 /usr/bin/python /usr/local/Cellar/couchdb-lucene/1.0.2/libexec/couchdb-external-hook.py

Indexing a document

I've created a new document over Futon and just pasted the following JSON in to it (you'll need to change javascript function to match parameters of your documents):

{
   "_id": "_design/search_station",
   "_rev": "1-e9b2c978433813a6d45010bac6de9971",
   "fulltext": {
       "by_name": {
           "index": "function(doc) { var ret=new Document(); ret.add(doc.name); return ret }"
       },
       "by_ip": {
           "index": "function(doc) { var ret=new Document(); ret.add(doc.ip); return ret }"
       }
   }
}

After that you should be able to run following CouchDb query:

http://localhost:5984/your-db-name/_fti/_design/search_station/by_name?q="Station1"
http://localhost:5984/your-db-name/_fti/_design/search_station/by_ip?q="192.168.3.14"

More

There is a lot more you can do with couchdb-lucene. For this please read the couchdb-lucene documentation

Installing Subversion with Apache on Debian

Pre-Conditions

This tutorial is written for Apache 2.2.22 and Subversion 1.6.17. This document will not be updated.

Install packages

As a first step we need to install all required packages.

sudo apt-get update
sudo apt-get install apache2
sudo apt-get install subversion
sudo apt-get install libapache2-svn

Include Apache modules

Here we activate some required apache modules

sudo a2enmod dav
sudo a2enmod dav_svn
sudo a2enmod ssl

Create directory structure

Next we create the directory structure.

  • public: Root directory of the virtual host container
  • repo: Directory that contains all subversion repositories
  • auth: Directory that contains all htpasswd files
  • ssl: Directory that contains the ssl cert and key
sudo mkdir /var/www/svn.yourdomain.com
sudo mkdir /var/www/svn.yourdomain.com/public
sudo mkdir /var/www/svn.yourdomain.com/repo
sudo mkdir /var/www/svn.yourdomain.com/auth
sudo mkdir /var/www/svn.yourdomain.com/ssl

Create repository

Next we create a first repository

sudo svnadmin create /var/www/svn.yourdomain.com/repo/test-project

Create group

Then we create the necessary groups

sudo groupadd subversion
sudo addgroup you subversion

Create credentials for login

As we authenticate with http basic method we create a password file

sudo htpasswd -c /var/www/svn.yourdomain.com/auth/test_project.passwd you

Create SSL certificate

When requesting a new certificate, it is possible to specify with -days how long the certificate will remain valid. In the example below it's 365 days. This will create both, the self-signed SSL certificate and the server key that protects it.

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /var/www/svn.yourdomain.com/ssl/ssl.key -out /var/www/svn.yourdomain.com/ssl/ssl.crt
After that a prompt appears that will ask us some questions.

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:New York
Locality Name (eg, city) []:NYC
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Your Inc
Organizational Unit Name (eg, section) []:Dept of something
Common Name (e.g. server FQDN or YOUR name) []:svn.yourdomain.com                  
Email Address []:webmaster@yourdomain.com

Create the virtualhost

Next we create a virtualhost entry: /etc/apache2/sites-available/svn.yourdomain.com.conf

<VirtualHost *:443>
  ServerName svn.yourdomain.com
  ServerAdmin webmaster@yourdomain.com
  ErrorLog /var/log/apache2/svn.yourdomain.com.err.log
  CustomLog /var/log/apache2/svn.yourdomain.com.acc.log combined
  DocumentRoot /var/www/svn.yourdomain.com/public
  <Directory "/var/www/svn.yourdomain.com/public">
    Order allow,deny
    Allow from all
  </Directory>

  SSLEngine on
  SSLCertificateFile /var/www/svn.yourdomain.com/ssl/ssl.crt
  SSLCertificateKeyFile /var/www/svn.yourdomain.com/ssl/ssl.key

  # Test project
  <Location /svn/test-project>
    DAV svn
    SVNPath /var/www/svn.yourdomain.com/repo/test-project
    AuthType Basic
    AuthName "Test-Project Server Repo"
    AuthUserFile /var/www/svn.yourdomain.com/auth/test-project.pwd
    Require valid-user
  </Location>
</VirtualHost>

Activate the virtualhost directive

Apache looks for files in /etc/apache2/sites-enabled. So you need to create a symlink.

cd /etc/apache2/sites-enabled/
sudo ln -s ../sites-available/svn.yourdomain.com.conf .

Set the right permissions

It is important that you set the right permissions so that apache can write data in to the svn repository and can also read the htpasswd file.

sudo chown -R www-data:subversion /var/www/svn.yourdomain.com

Reload Apache

service apache2 restart

Test Url

You should now be able to connect to https://svn.yourdomain.com/svn/test-project