Today’s changes – November 6, 2007

 

A few notable things we did in the last few weeks (for those keeping up with what we do):

  • Publigroupe and Swisscom announced the intension to intensify the online activities in the joint-venture. See press release here. What does that mean for local.ch in near future? You are going to see local.ch being transformed into a full-fledged white and yellow pages search engine. Further thanks to the collaboration with our partners – access to relevant local information today hidden behind walled gardens.

  • Autobörse and car4you joined team in print production. At your favorite kiosk you now find an extended Autobörse Magazine that additionally contains listings from car4you customers and interesting news on the car market.

  • Booking print listings into Autobörse and Fundgrueb got a lot easier – now you can choose between 3 different templates and simply pay with credit card (Visa & MasterCard). For now – only users with a user account have access to the product selection. Feel free to create a new account here.

  • Search for phone numbers now also returns entries with number ranges. Companies with a set of internal numbers – will now also be found. Good if you got a call from an unknown number – and you like to know who called you. Just type the phone number in the “What” field.

 
 

Using OpenSSL, RSA and RC4 to exchange encrypted data from PHP to Java

 

Needed a mechanism to be able to pass chunks of data securely from PHP to Java with requirements like;

  • If it accidentally ends up in a log file, it should be unusable
  • The system doing the encryption (web server, PHP) should be able to do so without exposing the means to decrypt the data i.e. we’re talking about some form of public-key cryptography.
  • On the PHP side, it needs to be fast and simple.
  • It shouldn’t result in significant extra work in maintaining public / private key pairs.

One solution might be something “from scratch” involving mcrypt or PHP libraries like Crypt_RSA. Do-able but more glue code you have, the greater the chance of unwittingly leaving security holes.

Another approach is GnuPG, either via the command line as discussed in this tutorial or via the GnuPG extension from PECL. While GnuPG is a solid way to do it, forking sub processes is ugly on public web servers and installing new extensions makes work. What’s more, we’re then have to maintain another set of public/private key pairs and we’re already using SSL…

Option 3 is using OpenSSL and PHP’s openssl_seal() function. SSL is normally used for encrypting networked communication between peers but that’s not all it can do. The OpenSSL extension has been a default part of the PHP distribution for some time now (since PHP 5.0.0 if I remember right), so no need to make work for sysadmins. It’s also a single function for the caller – fast and easy. What’s more – and perhaps the biggest win – it also allows us to re-use existing SSL certificates.

OpenSSL EVP

It’s worth having a grasp of what PHP’s openssl_seal() actually does – it’s really just a wrapper on OpenSSL’s EVP encryption envelope API. The way EVP works is something like this;

  1. It generates a random number (call it evk)
  2. It encrypts evk using a public key (e.g. from your web servers public SSL certificate) – call this Xevk. The key is encrypted using RSA. Note that though OpenSSL supports DSA certs, openssl_seal() doesn’t – tried it, got an error reporting RSA as required.
  3. Now it encrypt the data itself, using a symmetric key algorithm – the original evk is used as the key. Note the symmetric encryption alogithm is currently fixed as RC4 although the latest source suggests that may become something you can select in future PHP versions, using a fifth argument to openssl_seal().

The reason for using RSA and RC4 here is essentially because asymmetric encryption algorithms are generally expensive in terms of performance – you don’t want to encrypt big chunks of data this way or you’ll be in for a loooong wait. Instead RSA is used to encrypt only the key to the (much) faster symmetric algorithm. We then pass the encrypted data and the encrypted key to their recipient, where we have access to the private key available and can take the following steps;

  1. Load the SSL private key
  2. Decrypt Xevk using RSA and the private key to recover evk
  3. Decrypt the data using RC4 and key evk.

The PHP

The following example shows how it works in PHP using the OpenSSL extension API;


<?php
$dir = '/path/to/openssl/certificates';
$data = "A secret message for Bob"; // the data to encrypt

print "<pre>";
print "Input Data:\t$data\n";

// Get the public key
$publicKey = openssl_get_publickey(file_get_contents("$dir/pubkey_rsa.pem"));

// encrypt the data
openssl_seal($data, $sealed, $ekeys, array($publicKey));

openssl_free_key($publicKey);

$sealed = base64_encode($sealed);
$Xevk = base64_encode($ekeys[0]);

print "Encypted data:\t$sealed\n";
print "Xevk:\t\t$Xevk\n";

// Now do the round trip for example - decrypt using the private key
$privateKey = openssl_get_privatekey(file_get_contents("$dir/privkey_rsa.pem"));

openssl_open(base64_decode($sealed), $opened, base64_decode($Xevk), $privateKey)
or die(openssl_error_string());
openssl_free_key($privateKey);

print "Decypted data:\t$opened\n";
?>

Which, when executed, should give you output looking something like this;

Input Data:	A secret message for Bob
Encypted data:	E9u4eKgagNpScHUa9BzITQDJu72oGAf5
Xevk:		z81ZHFHc7pcmXRKM/7HulfFQvjPS9wZr63q1MgBMCcvYdVKmIA/FercSyeH6EZu8+APDDpq3kyUMLB1gW6UaRg==
Decypted data:	A secret message for Bob

Note that “Encypted data” and “Xevk” here are Base64 encoded and I can pass them around easily in this form.

The Java

To decrypt, the first working implementation looks like;


package ch.local.common.util.crypto;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileReader;
import java.security.Key;
import java.security.KeyPair;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.Security;
import java.security.cert.Certificate;

import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;

import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.openssl.PEMReader;
import org.bouncycastle.util.encoders.Base64;

/**
* For decrypting data encrypted with PHP's openssl_seal()
*
* Example - String envKey is the Base64 encoded, RSA encrypted envelop key
* and String sealedData is the Base64 encoded, RC4 encrypted payload, which
* you got from PHP's openssl_seal() then;
*
* <pre>
* KeyPair keyPair = OpenSSLInterop.keyPairPEMFile(
* "/path/to/openssl/privkey_rsa.pem"
* );
*
* PrivateKey privateKey = keyPair.getPrivate();
* String plainText = OpenSSLInterop.decrypt(sealedData, envKey, privateKey);
* </pre>
*
* @see http://www.php.net/openssl_seal
* @author Harry Fuecks
* @since Oct 25, 2007
* @version $Id$
*/
public class OpenSSLInterop {

private static boolean bcInitialized = false;

/**
* @param cipherKey
* @param cipherText
* @param privateKey
* @return
* @throws Exception
*/
public static String decrypt(String cipherText, String cipherKey, PrivateKey privateKey)
throws Exception {

return decrypt(cipherText, cipherKey, privateKey, "UTF-8");

}

/**
* @param cipherKey Base64 encoded, RSA encrypted key for RC4 decryption
* @param cipherText Base64 encoded, RC4 encrypted payload
* @param privateKey
* @param charsetName
* @return decrypted payload
* @throws Exception
*/
public static String decrypt(String cipherText, String cipherKey, PrivateKey privateKey, String charsetName)
throws Exception {

byte[] plainKey = decryptRSA(Base64.decode(cipherKey), privateKey);
byte[] plaintext = decryptRC4(plainKey, Base64.decode(cipherText));
return new String(plaintext, charsetName);

}

/**
* Loads a KeyPair object from an OpenSSL private key file
* (Just wrapper around Bouncycastles PEMReader)
* @param filename
* @return
* @throws Exception
*/
public static KeyPair keyPairFromPEMFile(String filename)
throws Exception {

if ( !bcInitialized ) {
Security.addProvider(new BouncyCastleProvider());
bcInitialized = true;
}

FileReader keyFile = new FileReader(filename);
PEMReader pemReader = new PEMReader(keyFile);
return (KeyPair)pemReader.readObject();

}

/**
* Returns a KeyPair from a Java keystore file.
*
* Note that you can convert OpenSSL keys into Java Keystore using the
* "Not yet commons-ssl" KeyStoreBuilder
* See - http://juliusdavies.ca/commons-ssl/utilities/
* e.g.
*
* $ java -cp not-yet-commons-ssl-0.3.8.jar org.apache.commons.ssl.KeyStoreBuilder \
* "privkey password" ./privkey.pem ./pubkey.pem
*
* @param filename
* @param alias
* @param keystorePassword
* @return
*/
public static KeyPair keyPairFromKeyStore(String filename, String alias, String keystorePassword)
throws Exception {

KeyStore keystore = KeyStore.getInstance("JKS");
File keystoreFile = new File(filename);

FileInputStream in = new FileInputStream(keystoreFile);
keystore.load(in, keystorePassword.toCharArray());
in.close();

Key key = keystore.getKey(alias, keystorePassword.toCharArray());
Certificate cert = keystore.getCertificate(alias);
PublicKey publicKey = cert.getPublicKey();

return new KeyPair(publicKey, (PrivateKey)key);

}

/**
* @param cipherKey
* @param privateKey
* @return
* @throws Exception
*/
private static byte[] decryptRSA(byte[] cipherKey, PrivateKey privateKey) throws Exception {

Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.DECRYPT_MODE, privateKey);
return cipher.doFinal(cipherKey);

}

/**
* Defaults to UTF-8 as the output encoding
* @param plainKey Base64 encoded, RSA encrypted key for RC4 decryption
* @param cipherText Base64 encoded, RC4 encrypted payload
* @return decrypted payload
* @throws Exception
*/
private static byte[] decryptRC4(byte[] plainKey, byte[] cipherText)
throws Exception {

SecretKey skeySpec = new SecretKeySpec(plainKey, "RC4");
Cipher cipher = Cipher.getInstance("RC4");
cipher.init(Cipher.DECRYPT_MODE, skeySpec);
return cipher.doFinal(cipherText);

}

}

Usage is simple…

KeyPair keyPair = OpenSSLInterop.keyPairPEMFile("/path/to/openssl/privkey_rsa.pem");
PrivateKey privateKey = keyPair.getPrivate();
String plainText = OpenSSLInterop.decrypt(sealedData, envKey, privateKey);

You could also use a keystore, for which the keyPairFromKeyStore() provides a short cut. Note that a fairly friendly way to convert PEM certificates to keystores is with Not Yet commons-ssl, using the KeyStoreBuilder e.g.

$ java -cp not-yet-commons-ssl-0.3.8.jar org.apache.commons.ssl.KeyStoreBuilder\
 "privkey password" /path/to/privkey.pem /path/to/pubkey.pem

In fact Not Yet commons-ssl also has a mechanism to decrypt ciphertexts generated by openssl’s password based encryption (PBE), used on the command line. It won’t work with output from PHP’s openssl_seal() though, because the PBE form makes use of a random salt, which it also places in the encrypted output.

The dependency on BouncyCastle (you need a bcprov[xxx].jar) is really only needed for reading .pem files – it can be eliminated fairly easily if you’re planning only on using Java keystores – you’d just need to use a different Base64 implementation.

By the way, no encyrpt method at this time because we don’t need it right now, but shouldn’t be too hard to implement.

 
 

We’re passionate about Geoweb

 

Today as a part of our Thursday Thirties series of talks, Cédric talks about the Geoweb.

                   Geoweb Talk by Cedric

Actually he gave a similar speech the other day at the BlogCamp Zürich and published the slides of his speech on his blog. Worth reading.

 
 

Today’s changes – October 16, 2007

 

We rolled out a platform release today – that kept us busy since the summer break. It includes search engine improvements in ranking and performance and a payment and order handling infrastructure.

What’s in for you:

  • The car search – especially the advanced search – got a redesign

  • Entries with extended content – like opening hours get featured in the search result – check this out

  • The guide now also finds events managed by the Le Temps newspaper team

 
 

As time goes by – we at local.ch

 

I felt the need to share some facts about how the local.ch team is doing and what we are working on.

Mentioning growth – we were able to welcome new friends to the engineering team – in the areas of mobile application development, geospatial expertise, linguistic processing and business rules & transaction automation. The research & development team consist of 13 engineers – additionally we have tightly integrated partners helping us in their core competencies like user experience, rich internet application development and information retrieval. We are looking forward to share some of our exciting developments in local search in the coming months.

Local.ch was started on the green field two years ago – today we are inherent part of PubliGroup and Swisscom (they are providing us with resources and access to vital parts of their core business). Part of the deal is that the new platform is replacing legacy systems and optimizes internal processes. As you probably know – it’s simpler to build something new than migrating existing stuff. Nevertheless, we spend and will spend considerable amount of time making sure the infrastructure is ready to improve todays yellow pages, phone book, guide and classifieds business and provide the framework to tab into future opportunities. And yes – we still have some of these mainframe beasts running in the cellar.

Where does that leave you? At local.ch we have the motto – “Users first” – translated to our business that means – empowering you to simply find relevant answers to local questions (wherever you are).

Examples:

  • Where is the nearest flower store that is still open in a hour?
  • Get me a dry-cleaner that is on my way to work.
  • Pizzerias serving after watching The Bourne Ultimatum?

Building software that can answer questions like this is one challenge, the other – far more complex – is having the accurate information. We are determined to build this piece by piece together with a series of premium partners.

Thank you for joining us.