Lennaert Ekelmans - LennieZ 4 months, 1 week, 1 day to my birthday!

Lennaert Ekelmans



Lennaert Ekelmans

I'm Lennaert Ekelmans, The Netherlands, 22 years old, serial entrepreneur, web developer, CakePHP'er, hobby-photographer and trance-addict.

Since April I've a blog, read here why.

How we built Twittermail.com in 48 hours

2009-03-24 01:00:00

Okay, for those who don't know Twittermail, Twittermail is an open idea by Boris Veldhuijzen van Zanten, he thought that it would be great if people could e-mail their tweets to Twitter.com.

This is potentially very useful for people who use mobile phones. Of course you can go to the mobile webpage of Twitter, but sending an e-mail is pretty much easier. Some older phones do not even contain a browser and only have e-mail functionality. Also a lot of businesses block Twitter.com on their corporate network but with Twittermail you can continually update your Twitter account even from your business e-mail address.

Those who register at Twittermail, get a secret unique email address like 1234abcde [AT] twittermail com. When you send an e-mail to this secret email address, it will get posted instantly to Twitter.com through the Twitter API.

The Environment

Twittermail is a small project so basically we only need a sign-up form, a settings page and we need to show some recent activity. Since we handle loads of visitors and more than 3000 emails a day this project needed to be very scalable and thin. We read a lot about the benchmarking results[sub]1[/sub]. We managed to do this project in exactly 48 hours, thanks to CakePHP (and Coca Cola).

Custom Validation

Users supply us their Twitter credentials so that we can create a unique email address. Because we need to verify if the information they provide is valid, we created a custom validation which checks the Twitter API whether or not everything is valid. In our model we created a custom function called 'isValidTwitter', in this function we make a CURL call to the Twitter API, we parse the output from JSON to an array and we get back the user information or an error.

 array(
            'rule' => array('isValidTwitter'),
            'message' => __('The credentials you supplied are not valid! Please try again')
        )
    );
?>
data[$this->name]['twitter_username'].":".$this->data[$this->name]['twitter_password']);
        curl_setopt($curl_twitter, CURLOPT_POST, true);
        curl_setopt($curl_twitter, CURLOPT_RETURNTRANSFER, true);
        $curl_result = curl_exec($curl_twitter);
        curl_close($curl_twitter);

        $twitterResponse = json_decode($curl_result, true);
?>

From now on you can do whatever you want with the results. In our project we encode the password, fetch some Twitter information and save it to the database.

Because we don't need a very complex User Authentication, we didn't use ACL.

Localization & Internationalization, l10n and i18n

At the moment our Website is English-only, but we already made it multi-language proof by using the __('Login') function. For those who never worked with localization, in the cake console you can simply generate a language file by using the command 'cake I18n' here you get the option to extract all __('') strings to POI files. You can read more about localization at the wonderful Cookbook at http://book.cakephp.org/view/162/Localizing-Your-Application

Scaling

Caching

Because we're currently serving more than 25,000 users and more than 1,000,000 emails have been sent, scaling is an serious subject. First of all, use Cache! You don't need to get all data over and over again from the database. For example in the sidebar we have 'Newest Twittermailers'. This is cached for a year (yes, a year), every time when someone is registering, we're resetting the cache. Cache isn't a thing you should forget, and for those who never worked with it please read the Cookbook.

// Reset / Delete the cache, so next time the newest Twittermailers list will be re-generated.
Cache::delete('sidebar_newesttwitters');

Containable Behavior

This is a very substantial improvement in the Cake 1.2 series, in the past I always used for my CakePHP projects 'expect' (For 1.1 users: http://bakery.cakephp.org/articles/view/keeping-bindmodel-and-unbindmodel-out-of-your-controllers). This model behavior allows you to filter and limit model find operations. Using Containable will help you cut down on needless wear and tear on your database, increasing the speed and overall performance of your application. The class will also help you search and filter your data for your users in a clean and consistent way.

It helps you to select only the data you really want, it's recommended to read the full recept about this at http://book.cakephp.org/view/474/Containable

Think about security

Often developers forget to think about security, but security is a very important subject when you're developing web-applications. Use the validations and double check them with the sanitize object built-in CakePHP.

App::import('Sanitize');

Put this code in your AppController and AppModel to achieve a higher lever of security, read about it at http://book.cakephp.org/view/153/Data-Sanitization/ For example:

echo Sanitize::html($untrustedString, true);

And remember: NEVER trust the input of your users!


Then there's the Model::save function, that lacks a complete description in the cakebook, but in my opinion the $fieldList method is very important.
Like I said before, never trust the input of your users, supply $fieldList with an array of all the fields you're supposed to fill.

For example:

$this->Twitter->save($this->data, true, array('username', 'password', 'email'));

How we did it in 48 hours


f course, this is a (very) small project, but because of Cake we were able to manage this project in 48 hours. It's essential to stick to the CakePHP 'rules', just use the validation methods, use the helpers, use the build-in components, it actually helps you with keeping your code clean and compact.

Some respect


Show the world you're happy with CakePHP, put the 'cakephp - power' button on your website, be proud you're using CakePHP. Also don't forget to donate, the cake developers team are working day-and-night to supply us a better CakePHP, the only thing we could do is using CakePHP and show our satisfaction, at the end of 2009 we will donate the Cake Foundation 10% of Twittermail's profit.


Happy baking,

Bram Kok
Lennaert Ekelmans
http://www.twittermail.com

Giuliano Barberi Giuliano Barberi on 28/3/09
Very cool.

How experienced were you guys with Cake prior to using it for this project? And how has it handled the traffic (did you have to install more hardware than before)?

Regards!
Thomas Pedersen Thomas Pedersen on 28/3/09
Won't you be infringing twitters trademark by having a domain name and logo like that?
Oliver John Tibi Oliver John Tibi on 28/3/09
Well done! Did you have any custom component to generate the secret email addresses for the users? It would be nice if you'd share them too. :)
Lennaert Ekelmans Lennaert Ekelmans on 28/3/09
@Giuliano Barberi: We were kinda experienced, we built 3 others projects in CakePHP, but of course we have still much to learn.

Surprisingly we didn't had to add extra hardware, with CakePHP the serverload was incredible less than before. I don't really know why, probrably because of the intelligent queries and caching.

@Thomas Pedersen: We were one of the first Twitter applications were it was legal to have twitter in the domainname. So I guess it's no big problem, we like Twitter and Twitter likes us.

@Oliver John Tibi: Well Oliver, we just used a small snippet to generate a random code, here you see:


function _generateCode()
{
$karakters = "abcdefghjkmnpqrstuvwxyz23456789";

$randkey = "";
$max=strlen($karakters)-1;
for ($i=0;$i<6;$i++) {
$randkey .= substr($karakters, rand(0, $max), 1);
}

// Check if the secret code already exists
$findSecret = $this->find('first', array(
'conditions' => array(
'secret_twittermailcode' => $randkey),
'fields' => array('id', 'secret')
)
);

// Already exists?
if($findSecret) {
return _generateCode();
} else {
return $randkey;
}
}
Sint Smeding Sint Smeding on 1/4/09
The whole project has a lot of 'Getting Real' philosophy in it. Did you read the book? I can really recommend reading that one!
Lennaert Lennaert on 11/4/09
@Sint, No I haven't read it. I will take a look, it seems interessesting.

Write a comment

I won't publish your e-mail address, website is optional