ELCODOlog

GG.pl REST API: python-ggapi

Tue, 07 Dec 2010 08:00:00 +0100

Yesterday, we’ve released simple implementation of GG.pl API. GG.pl is Facebook-like site based on most popular polish IM - Gadu-Gadu.

python-ggapi is a client library that allows you to send notifications to users, get their data from public directory, and post to dashboard.

More about GG.pl and it’s API: http://dev.gg.pl (in polish).

Library is licensed under OSI BSD license. For manual and download check github: https://github.com/elcodo/python-ggapi

Wczoraj wydaliśmy pierwszą wersję modułu Python obsługującego API serwisu GG.pl.

Pozwala on na pobieranie danych z katalogu publicznego, wysyłanie notyfikacji do użytkowników, umieszczanie wiadomości na pulpicie użytkownika.

Więcej o API GG.pl: http://dev.gg.pl

Dokumentację i kod źródłowy (licencja BSD) można znaleźć na github: https://github.com/elcodo/python-ggapi

Tumblr: QuickTumble Safari Extension

Wed, 18 Aug 2010 19:44:03 +0200

We made simple extension for Safari 5 which allows you to quick post to tumblr. It is called QuickTumble and can be obtained here for free.

Currently it provides:

toolbar button - toggles bar with shortcuts to add different media types to tumblelog (text, photo, link, etc.)
“Tumble Link to this Page” context menu for quick posting links to currently browsed page
“Tumble Selection as Quote” context menu for quick posting selected text on visited page as quote

For screenshots and download link please visit QuickTumble extension page.

We love to hear your opinion about it! Please post comments here, on Facebook or on our Twitter (@ELCODOsoft). Tell us what you would add, change or remove.

Django: detect user's first login

Mon, 09 Aug 2010 14:39:38 +0200

Sometimes we want to detect information about user’s first login, e.g. to show account detail or edit form.

First thought would be to look at last_login field in User model, but… it is filled with date after user signed up and doesn’t logged in yet.

So, can we assume that User.last_login and User.date_joined fields have equal value after registration? I think it is too risky - dates might be equal but it is not guaranteed.

In this case we must extend our user profile model (more on this topic: http://docs.djangoproject.com/en/dev/topics/auth/#storing-additional-information-about-users). This is not the cleanest method but the simplest one.

Let’s add new field to out UserProfile model:

class UserProfile(models.Model):
    """Additional user fields"""
    user = models.ForeignKey(User, unique=True)    
    (...)
    is_first_login = models.BooleanField(default=True,
        verbose_name=_(u"Is first login?"))

Okay, now we need to detect user login and check if it is his first attempt. We will create custom middleware to do the check.

Middleware should be located in settings.py’s MIDDLEWARE_CLASSES after ‘django.contrib.auth.middleware.AuthenticationMiddleware’:

MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'users.middleware.FirstLoginMiddleware',
    (...)
)

This is FirstLoginMiddleware (e.g. users.middleware.FirstLoginMiddleware):

# -*- coding: utf-8 -*-
from django.core.urlresolvers import reverse
from django.http import HttpResponseRedirect


class FirstLoginMiddleware(object):

    def process_request(self, request):
        if request.user.is_authenticated() and "dont_check_first_login" \
            not in request.session:
            request.session["dont_check_first_login"] = True
            profile = request.user.get_profile()
            if profile.is_first_login:
                profile.is_first_login = False
                profile.save()
                return HttpResponseRedirect(reverse('users:account_edit'))

As you can see, we used session variable named “dont_check_first_login” (long name, isn’t it?).

It is used to determine that user has already logged in so we don’t query database in each request. Consider caching user profile in real life app - this code is example how to catch first login.

Facebook: Profile tab with your GitHub repositories

Wed, 04 Aug 2010 11:21:00 +0200

We made simple application for Facebook allowing you to display your open source (public) repositories stored on GitHub. Obviously, it’s called GitHub Repositories and is available here: http://www.facebook.com/apps/application.php?id=120729024640004.

Here’s how it looks:

For live demo go to our Facebook page.

It is very simple at the moment - when you connect app to your profile it will ask for username, then you could add tab to your profile using “+”.

We will continue to develop it so GitHub user profile data is displayed and you could change username without having to uninstall app and install it again.

Feel free to post ideas, bugs, anything on Application Page. If you like GitHub Repositories - install it on your profile and/or like it on Facebook! :)

Django: How to see if model instance was changed after save (or how to keep model instance state)

Sat, 17 Jul 2010 23:48:00 +0200

There are situations when you want to know when model instance was changed. E.g. slugs generation when you change title (or maybe this example is discussable when comes to SEO?).

SEO matters but there are some real-life situations when you need this - let’s look at following example:

- you’re using django-mptt or some other method to store your document tree,

- URL to your document is generated using all ancestor names like: “Document” with parent “Data” has an URL like this: /data/document/

- you want to rebuild all children URLs when root or any parent document title/slug changes.

Nevertheless, there is some very simple solution to detect model state.

In your models.py, add new property like that:

class ExampleModel(models.Model):
    # <>
    before_save = None

Now, it’s time for signals - we will use pre_save and post_save. First one for getting before save data, second one to recognize it and do some action.

It is easy to detect field value change, e.g. title change generates new slug. See following example:

def object_pre_save(instance, **kwargs):
    """Store object state before save"""
    try:
        instance.before_save = ExampleModel.objects.get(pk=instance.pk)
    except Transaction.DoesNotExist:
        instance.before_save = None

def object_post_save(sender, instance, created, **kwargs):
    if instance.before_save.title != instance.title:
    # <>
    instance.save()

pre_save.connect(object_pre_save, sender=ExampleModel)
post_save.connect(object_post_save, sender=ExampleModel)

Of course, you can do more complex lookups for change since there is complete “before-save” instance avaible at instance.before_save.

Okay, it is using extra query but if you do it right - e.g. in admin it shouldn’t kill your service. Maybe you have better solution for checking if data has changed? Let us know!

Django: register user with invitation code

Wed, 14 Jul 2010 11:26:32 +0200

Django: register user with invitation code:

We made custom backend for django-registration app allowing you to simplify your service beta testing by require an invitation code from user.

Try it, it’s open source and free to fork and modify!

Remember that generation and distribution of codes is up to you.

Django/OpenSource Scalability

Wed, 30 Jun 2010 01:08:00 +0200

Django is great. No doubts about it. But - sometimes comes the question of scalability, especially in a time-and-budget constrains. Well - the very least we can say is that - our client suddenly decided that he needed about 200 rqs - yes, that’s 200 hundred requests per second - and yes, that’s 1728.000 requestes per day. Well, no problem in achieveing that - so far there’s no talk about budget. Well, there was, the all-dreaded least-cost-budget-criteria. So, ELCODO team stood up for the chalenge and came up with a few interesting, open-source ideas. First of them was the load balancing - here came the ideas about using vrrp/carp. The only real concern was about the instability of the IPv6 implementation - which was easily resolved through aggresive negotiations with the client :)

The second issue was of similar scale - a choice between using replication of the MySQL database and using a dedicated MySQL server. The dedicated option was chosen. So no architectural or otherwise programistic problems here. One database server and that’s enough for now.

The first of real problems came with the Django module consisting of some sort of an upload module - the uploaded files needed to be in an exact sync with every server.

For example - server1.elcodo.pl needed to have the same files ijn /upload/ dir as server2.elcodo.pl. The problem was really easy to resolve - given the use of a replicated, clustered filesystem such as GlusterFS. Under consideration, GlusterFS was the most robust, feature-full choice we could make in under a time limit of several minutes. So, a GlusterFS filesystem was created, using 4 servers as filesystem nodes.

The last problem is considered by many a no-problem situation, but for us is a really headache-case. The problem of “overall efect”, the situation in which you just spent 8 hourse building the systems and it’s simply too slow. The GlusterFS worked like a charm, giving us near real-time results. The database (not replicated) was working fine, as far as the MySQL settings allowed it). One would suggest that the only problem (besides implementing VRRP/Carp in a reasonable way) was the webserver himself. That is not true. We have considered an advanced Nginx (and eventually a Litghttpd) setup - but the Apache 2.2.* was enough. Using the mod_wsgi module without daemon process seems to be a preety acceptable solution.

So, to bring some “hard-evidence” and “true statistics”:

- you can easily serve ~200++ requests/sec with Apache/mod_wsgi on cheap hardware

- Networked, replicated filesystems - like GlusterFS - are really neat

- all you need is a good LoadBalancing. Good LB will get you about 70% problems less

- Have a fat pipe if you want to deliver “fat” content - no cache or otherwise cheating will help you.

Summing up - 4 servers + Django (mod_wsgi) + MySQL + GlusterFS + LoadBalancing.

3 hours of work. Hardware/instalation cost below 1000$. And that’s called OpenSource ;)

brutto: The more focus on profit, the less good ideas - Social...

Fri, 28 May 2010 11:28:17 +0200

brutto:

The more focus on profit, the less good ideas - Social Hallucinations via ahajduk

jednym tchem

seapomeranian: mongoengine vs. django-mongokit - Peterbe.com...

Tue, 25 May 2010 14:35:05 +0200

seapomeranian:

mongoengine vs. django-mongokit - Peterbe.com (Peter Bengtsson on Python, Django, Zope, Kung Fu, London and photos)

Django: the easiest way to run manage commands in crontab

Tue, 25 May 2010 11:28:44 +0200

Sometimes there are some management commands I would like to run from crontab at given time or interval. There are some ways when you import setup_environ from Django but there is much simpler solution.

For example - Haystack search index update. Let’s do it in one hour interval.

user@server:~$ crontab -l
PYTHONPATH=":/home/www/project/django/lib/python2.5/site-packages"
DJANGO_SETTINGS_MODULE=project.settings

# m h  dom mon dow   command
0 * * * * /home/www/project/manage.py update_index

So, all you need is to set PYTHONPATH and DJANGO_SETTINGS_MODULE according to your settings. And the trick is that it can be done in crontab.

Note that this was tested on Debian Linux.

Mac OS X: ⌘ + ⇥ and Dock stops working

Thu, 06 May 2010 22:25:00 +0200

If your Dock stops responding (e.g. cannot click application icon) and you can’t switch between applications using ⌘ + ⇥ (Command + Tab) there is an easy way to deal with it without restart.

Just open Activity Monitor (typically ⌘ + ␣), find Dock process and kill process using “Quit Process” option. Dock should be restared after a second and everything should back to normal.

Django: how to mark required form field with *?

Wed, 28 Apr 2010 11:31:30 +0200

If you want to mark required fields with * or some other text/symbol - don’t use some hacks in your python code (Label modify in form class, etc.). Things like that should reside in templates and all you need to do is to apply this:

{% if field.field.required %} *{% endif %}

E.g. form display template:


{% csrf_token %}
{% for hidden in form.hidden_fields %}
	{{ hidden }}
{% endfor %}


{% for field in form.visible_fields %}
	{% if field.errors %}
		 
		{{ field.errors }}
	{% endif %}
	
		{{ field.label_tag }}
		{% if field.field.required %} *{% endif %}:
	
	{{ field|safe }}
{% endfor %}

"We will either find a way, or make one."

Tue, 13 Apr 2010 23:58:00 +0200

“We will either find a way, or make one.”

- Hannibal