Planet TurboGears

Last week Science Daily published an article that caught my attention titled 'Sounds of Silence' Proving a Hit: World's Fastest Random Number Generator. The tl;dr is that researchers at the ANU ARC Centre of Excellence for Quantum Computation and Communication Technology created a blazing fast random number generator based on quantum fluctuations in a vacuum. Thankfully, these awesome scientists are giving their data away for free, and they even provide a JSON API.

In an effort to make it simple to leverage this data, I created a new project: quantumrandom. It provides a qrandom command-line tool, a Python API, and also a /dev/qrandom Linux character device.

Installing

$ virtualenv env
$ source env/bin/activate
$ pip install quantumrandom

Using the command-line tool

$ qrandom --int --min 5 --max 15
7
$ qrandom --binary
���I�%��e(�1��c��Ee�4�������j�Կ��=�^H�c�u
oq��G��Z�^���fK�0_��h��s�b��AE=�rR~���(�^
+a�a̙�IB�,S�!ꀔd�2H~�X�Z����R��.f
...
$ qrandom --hex
1dc59fde43b5045120453186d45653dd455bd8e6fc7d8c591f0018fa9261ab2835eb210e8
e267cf35a54c02ce2a93b3ec448c4c7aa84fdedb61c7b0d87c9e7acf8e9fdadc8d68bcaa5a
...

Creating /dev/qrandom

quantumrandom comes equipped with a multi-threaded character device in userspace. When read from, this device fires up a bunch of threads to fetch data. Not only can you utilize this as a rng, but you can also feed this data back into your system's entropy pool.

In order to build it's dependencies, you'll need the following packages installed: svn gcc-c++ fuse-devel gccxml libattr-devel. On Fedora 17 and newer, you'll also need the kernel-modules-extra package installed for the cuse module.

pip install ctypeslib hg+https://cusepy.googlecode.com/hg
sudo modprobe cuse
sudo chmod 666 /dev/cuse
qrandom-dev -v
sudo chmod 666 /dev/qrandom

By default it will use 3 threads, which can be changed by passing '-t #' into the qrandom-dev.

Testing the randomness for FIPS 140-2 compliance

$ cat /dev/qrandom | rngtest --blockcount=1000
rngtest: bits received from input: 20000032
rngtest: FIPS 140-2 successes: 1000
rngtest: FIPS 140-2 failures: 0
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 0
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 0
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=17.696; avg=386.711; max=4882812.500)Kibits/s
rngtest: FIPS tests speed: (min=10.949; avg=94.538; max=161.640)Mibits/s
rngtest: Program run time: 50708319 microseconds

Adding entropy to the Linux random number generator

sudo rngd --rng-device=/dev/qrandom --random-device=/dev/random --timeout=5 --foreground

Monitoring your available entropy levels

watch -n 1 cat /proc/sys/kernel/random/entropy_avail

Python API

The quantumrandom Python module contains a low-level get_data function, which is modelled after the ANU Quantum Random Number Generator's JSON API. It returns variable-length lists of either uint16 or hex16 data.

>>> quantumrandom.get_data(data_type='uint16', array_length=5)
[42796, 32457, 9242, 11316, 21078]
>>> quantumrandom.get_data(data_type='hex16', array_length=5, block_size=2)
['f1d5', '0eb3', '1119', '7cfd', '64ce']

Based on this get_data function, quantumrandom also provides a bunch of higher-level helper functions that make easy to perform a variety of tasks.

>>> quantumrandom.randint(0, 20)
5
>>> quantumrandom.hex()[:10]
'8272613343'
>>> quantumrandom.binary()[0]
'\xa5'
>>> len(quantumrandom.binary())
10000
>>> quantumrandom.uint16()
numpy.array([24094, 13944, 22109, 22908, 34878, 33797, 47221, 21485, 37930, ...], dtype=numpy.uint16)
>>> quantumrandom.uint16().data[:10]
'\x87\x7fY.\xcc\xab\xea\r\x1c`'

Follow quantumrandom on GitHub: https://github.com/lmacken/quantumrandom

Many beginner web developers ask questions like “how do i do something without refreshing page?” or “how can i load another page in my page?” etc. The answer is AJAX. AJAX stands for “Asynchronous JavaScript and XML”. With AJAX, you can retrieve or send data to the server in the background using an “XMLHttpRequest.”

Before going further, i would like to take your time and advise you to learn JavaScript. If you are a web developer, no matter if you are back-end or front-end one, at some point in your career you will have to use JavaScript. In order to be ready for those times, go and learn JavasSript. There are awesome resources.

Do not be one of those people who think learning jQuery is enough and how you don’t need to know JavaScript. jQuery is a JS framework. It’s a great one and that’s all. Learn JavaScript.

There are many techniques to use AJAX in a web application. In these examples we are going to use jQuery framework and it’s powerful “$.get” and “$.post” methods.

Open up your terminal, cd into your TurboGears environment, activate it, upgrade it to the most recent version which is 2.1.5 with:

easy_install -U -i http://tg.gy/current tg.devtools

Go to your controllers/root.py and change the index action like this:

@expose('ideas.templates.index')
def index(self):
    """Handle the front-page."""
    projects = DBSession.query(Project).all()
    return dict(page='index', projects=projects)

And this is our new “templates/index.mak”:

<%inherit file="local:templates.master"/>

<%def name="title()">
  Welcome to IDEAS.
</%def>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
<script type="text/javascript">
$(function(){
    $("a.project_link").click(function(){
        var context = this;
        $.get(this.href, function(response){
            $(context).parent().next().html(response);
        });
        return false;
    });
});
</script>

<div style="margin-bottom: 50px;">
    <h1>Projects</h1>
    % for project in projects:
        <div class="project">
            <div style="margin-bottom: 5px;"><a class="project_link" href="${url('/show_tasks/%s' % project.id)}">${project.project_name}</a></div>
        <div class="tasks"></div>
    % endfor
</div>

<%def name="sidebar_bottom()"></%def>

In our “show_tasks.mak” template:

• We are listing the projects in a for loop.
• We create a “div” element with the class “tasks” to use in our JavaScript.
• We include jQuery framework via Google CDN.

So whenever we click to a project’s link, its tasks will be listed under the “tasks” div.

Let me walk you through the simple JavaScript codes we wrote.

In the first line, we tell jQuery “when the DOM is ready”, use all these child scripts of yours. So it attaches the event listener, changes css classes, anything we want to do.

In the second line, we tell, whenever an “a” element with a “project_link” class is clicked, create an AJAX request and append the result to the closest div with the class “tasks” to the “a” element’s parent.

In JavaScript, whenever a new scope is opened like the second line, the keyword “this” changes meaning. In line 3 of our script, it references to the clicked “a” element however in the line 5 it references the response object. This is why I have created the “context” variable and mapped it to “this” which is the clicked “a” element. This way, I am able to use the clicked a element.

We are missing two things here. The corresponding TurboGears action and it’s template. So let’s create them. In your RootController, add the following action:

@expose('ideas.templates.show_tasks')
def show_tasks(self, project_id):
    tasks = DBSession.query(Task).filter_by(project_id=project_id).all()
    return dict(tasks=tasks)

And the following goes by “show_tasks.mak” under the “templates” directory.

% if tasks:
    % for task in tasks:
        <div>${task.task_name}</div>
    % endfor
% else:
    <div>This project does not have any tasks, yet.</div>
% endif

That’s all. Run your server and open it up in your browser. Let’s see how it goes.

In the next post, I will show you how to use the $.post method and deal with JSON responses from the server when using AJAX. I hope you guys enjoyed all these. Please do not hesitate to ask any question. You are more than welcome.

TG 2.2 beta has been released and with it many improvements, I was quite curious about the results of the performance enhancements derived of moving TG to Crank, I do not expect to see a big improvement as this is only a Beta and there is still many things to do and tweak before final release, also the real performance work will come from the 2.3 branch, but for now I would like to post my benchmark results.

I’m using the same benchmarking method as I used to tests the templating systems, this is now easier as 2.2 integrates many jinja improvements, the difference this time is that I will be benchmarking the build-in Paster server vs Gunicorn vs UWSGI, lets see how it goes.

Paster

As we can see templating performance status quo is more or less the same, Genshi is still slow but there has been good improvements, the environ page rendering is now as fast as other templates, but overall is Genshi in 2nd and Jinja and Mako more or less tied in 1st place.

Overall performance has improved in comparison with 2.1 as you can see, ~20 rps more has been squeezed in this release.

Gunicorn

gunicorn_paster 0.14.2 was used in this test, running with 2 sync workers.

With Gunicorn we can see that the Genshi performance on the environ rendering test is no fluke, here Jinja is only a bit faster than Mako, overall speed wise 2.2 Beta is very similar to 2.1 under Gunicorn but Mako is now faster (as fast as Jinja) and Genshi has got a nice speedup too.

uWSGI

uwsgi 1.1.2 was used for this test, with 2 workers, with Master option (-M), using –http :8080 option to serve directly instead of using uwsgi protocol (to serve under nginx).

Finally the uWSGI server, here we can see TG 2.2 shine, uWSGI is a bit faster overall than Gunicorn, templating performance is similar in this test, Jinja and Mako are more or less tied.

Closing

As we can see Turbogears 2 is pretty much alive and kicking, development is going to ramp up pretty soon as tw2 sprints and migration has keeped the TG team busy but that is mostly done, it would be also awesome if people could give TG 2.2 Beta a try and report problems, to do so install as per TG Instructions but use the beta package repository instead of the regular one:

easy_install -i http://tg.gy/beta tg.devtools

Internet in Mexico is something relatively new, I recall 10 years ago we had dial-up, then a few years later ADSL went live and we had 256kbps, the ISP has been steadily increasing its capacity and plans, the lowest bandwidth plan is now 1mbps, I have installed in my home 2mbps adsl line with phone line which runs around 50dlls monthly.

Now talking about prices, according to OECD, Mexico has one of the most expensive internet services of its member countries, and as you can see they are right, 50 dollars for only 2mbps is practically unheard from any other country on the OECD, this is very much attributed to the phone company Telmex, which is currently one of the biggest monopolies in our country from one of the richest man in the world (Carlos Slim).

This has been changing a bit lately, a few months ago my speed boosted from 2mbps to 20mbps, wow!, 20mbps for 50dlls with phone line!, that is a game changer!, we practically went up to being competitive on the OECD tables, but of course I got disappointing a few days ago, speeds got back to 10mbps, then to 5mbps where they are actually, 5 mbps for 50 dollars is still one of the most expensive internet services according OECD.

All in all we know monopolies are detrimental to a free capitalist economy, and Mexico is really full of monopolies, raging from government held monopolies (some really silly like electricity generation), to private monopolies (Telmex), duopolis (Azteca TV, Televisa), even at municipal scale there are monopolies which really impact negatively like the local transportation, bus transportation in the city where I live is a concession from the government to a single group, now we have problems with a transport system that is inefficient, in bad conditions and expensive, corruption in Mexico really aggravates this problem, take for example local transportation, if another group wanted to enter the fray and offer better and cheaper services it would have to acquire a concession from the government, which will surely reject them, this happens at all levels, phone and ISP competitors arevery new thanks to Telmex colluding with the government to not let other companies enter, even now any new company to enter the telecommunications segment has it very hard.

In the end ranting is mostly the only thing we can do, as even trying to break monopolies by opening your own company is not possible in this country, fortunately there are still many other segments which are not occupied by monopolies and where entrepreneurs can get in a grow.

I have always seen a lot of people speak highly of Mac OSX as a development platform and Apple hardware as a high quality platform in general, lately I have been in charge of some mobile development, a small OpenGL app, which I initially wrote on my linux machine taking care of making it very cross platform, then it came the time to port to IOS, the port was pretty simple and most things worked out of the box without much trouble.

As much as it sounds like a story with a happy ending I got to say it was a short but hellish walk when porting, stuff mostly worked ok, there where things like OpenAL that did not work as expected and required a bit of extra work, but the one thing that drove me mad was the general OSX interface, not the graphical interface per see, but the details that where totally different from what I was used to, like keybindings, for example, when editing code, I expect the End key to go to the end of the line, on OSX it just scrolls at the end of the file without moving the cursor, stuff like that really trowed me out of balance and maked developed way slower than it should and very painful, while I do not doubt you can get accustomed to OSX but coming from other OSes where stuff work a way (which by the way works exactly the same even on Windows and other OSes) and then having it all flipped it just pisses me of and lowers my productivity.

Personally I would not have an Apple machine as my main development workstation or laptop, I’m already too accustomed to Linux and its quirks, OSX is a whole new set of quirks to workout on a platform which is less flexible to me for a relatively little gain.

If you remember the post that we created the models in, you will also remember that TurboGears uses SQLAlchemy for the model part of MVC.

Now, we are going to do some practice on querying the models. In the meanwhile, you will discover the TurboGears console. Let’s get into our project’s home dir and start the console.

cd ~/projects/tg2env/ideas
source ../bin/activate
paster shell development.ini

Let’s import our models and the DBSession object from ideas/model. At first, we are going to create some projects and tasks. When using queries, you will see which queries are running.

from ideas.model import DBSession, Project, Task
p1 = Project(project_name="TurboGears Shell Project")
p2 = Project(project_name="TurboGears Blog Project")
DBSession.add(p1)
DBSession.add(p2)
DBSession.flush()
t1 = Task(project_id=p1.id, task_name="Extend the shell", status="open")
t2 = Task(project_id=p2.id, task_name="Create the models", status="closed")
t3 = Task(project_id=p2.id, task_name="Use Twitter Bootstrap in the templates", status="open")
DBSession.add(t1)
DBSession.add(t2)
DBSession.add(t3)
DBSession.flush()

We have created two projects, the first project has 1 open task, second project has 2 tasks; 1 open and 1 closed. We are going to start with the basics.

As the first step, let’s query all projects.

DBSession.query(Project).all()
# output:
00:30:09,385 INFO  [sqlalchemy.engine.base.Engine] SELECT projects.id AS projects_id, projects.project_name AS projects_project_name, projects.project_description AS projects_project_description 
FROM projects
00:30:09,385 INFO  [sqlalchemy.engine.base.Engine] ()
[<ideas.model.project.Project object at 0x4161bd0>, <ideas.model.project.Project object at 0x40982d0>, <ideas.model.project.Project object at 0x40a42d0>]

I had 3 projects in my “ideas” app so it returned all 3.

Let’s order the projects by their name in the ascending order.

projects = DBSession.query(Project).order_by(Project.project_name.asc()).all()

>>> for p in projects: print p.project_name
... 
TurboGears Blog Project
TurboGears Project Management
TurboGears Shell Project

Now we are going to list all open tasks of “TurboGears Blog Project”.

DBSession.query(Task).filter(Task.project_id == p2.id).filter(Task.status == "open").all()
>>> for task in tasks: print task.task_name
... 
Use Twitter Bootstrap in the template

SQLAlchemy has an expression called “and_” so we can write the above query like this:

>>> from sqlalchemy.sql.expression import and_
>>> DBSession.query(Task).filter(and_(Task.project_id == p2.id, Task.status == "open")).all()
>>> for task in tasks: print task.task_name
... 
Use Twitter Bootstrap in the templates

And we are going to update the “Use Twitter Bootstrap in the template” task and set its status to “closed”.

DBSession.query(Task).filter(Task.id == t3.id).update({"status": "closed"})
00:45:53,126 INFO  [sqlalchemy.engine.base.Engine] UPDATE tasks SET status=? WHERE tasks.id = ?
00:45:53,126 INFO  [sqlalchemy.engine.base.Engine] ('closed', 4)
>>> t3.status
'closed'

And let's delete all the tasks of "TurboGears Shell Project".

DBSession.query(Task).filter_by(project_id=t1.id).delete()
00:59:01,585 INFO  [sqlalchemy.engine.base.Engine] DELETE FROM tasks WHERE tasks.project_id = ?
00:59:01,585 INFO  [sqlalchemy.engine.base.Engine] (2,)

For more on SQLAlchemy, please go read on http://docs.sqlalchemy.org/en/latest/orm/query.html and you can do the Object Relation Tutorial.

We have created our models, created our controller actions and forms. Now we are going to set up our templates. Let’s take a look at the controller action “new_project”:

@expose('ideas.templates.new_project')
def new_project(self):
    """ Displays the project form. """
    return dict(project_form=project_form)

In this action, we tell TurboGears that when new_project action is called, we are going to render it with “new_project.mak” file in the “templates” folder. We also pass “project_form” variable to the template.

Let’s create our templates. This is the content of new_project.mak:

<!-- ideas/templates/new_project.mak -->
<%inherit file="local:templates.master"/>

<%def name="title()">Project Form</%def>

${project_form()|n}

This is the content of new_task.mak:

<!-- ideas/templates/new_task.mak -->
<%inherit file="local:templates.master"/>

<%def name="title()">Task Form</%def>

${task_form()|n}

And this is the content of show_project.mak:

<!-- ideas/templates/show_project.mak -->
<%inherit file="local:templates.master"/>

<%def name="title()">${project.project_name}</%def>

<h1>${project.project_name}</h1>
<div>${project.project_description}</div>

<h2>Tasks</h2>
<ul>
    % for task in tasks:
        <li>${task.task_name}</li>
    % endfor 
</ul>

Now start your web server and visit “http://127.0.0.1:8080/new_project” and see how the project creation form looks like:

And let’s head to task creation page at “http://127.0.0.1:8080/new_task“:

Go ahead and play with it. Do something yourself. For more on the templates you can visit mako documentation.

A new era for TurboGears has started.

From core developers to community members, we have started doing something for TurboGears. It deserves it. We love it. We want to spread it. That's why, if you are wondering.

I have started a blog on TurboGears development. For you. For any web developer. For any Python web developer. You will start from scratch and become an advanced TurboGears developer. You will like it. You will become very productive with TurboGears.

You can visit the blog at http://tglogs.wordpress.com.

Let me know your ideas, your suggestions and feedback.

Enjoy!

In the previous post, we have created our Project and Task models. Now we have to create our forms.

It is very easy to create forms with TurboGears. In this example we are going to use ToscaWidgets 1 however keep in mind that in the next major TurboGears release, TurboGears will switch to ToscaWidgets 2.

HTML forms consist of a form element and its child elements called input. In these examples, we are going to create forms in Pytho thon with the help of ToscaWidgets library and render these forms in our templates.

This is a simple example of creating a form.

# ideas/ideas/forms/project_form.py
# -*- coding: utf-8 -*-
"""Project form module"""

from tw.api import WidgetsList
import tw.forms as forms
from ideas.model.project import Project

class ProjectForm(forms.TableForm):
    class fields(WidgetsList):
        project_name = forms.TextField(validator=forms.validators.NotEmpty)
        project_description = forms.TextArea(validator=forms.validators.NotEmpty)

class TaskForm(forms.TableForm):
    class fields(WidgetsList):
        project_id = forms.SingleSelectField(options=Project.get_projects_for_dropdown())
        task_name = forms.TextField(validator=forms.validators.NotEmpty)
        task_description = forms.TextArea(validator=forms.validators.NotEmpty)
        status = forms.RadioButtonList(options=(('open', 'Open'), ('closed', 'Closed')))

project_form = ProjectForm(action="create_project")
task_form = TaskForm(action="create_task")

1) We have created two classes called ProjectForm and TaskForm inheriting ToscaWidgets’ TableForm class. When we call the project_form and task_form in our templates, it will render the form elements in a table layout.

2) In the ProjectForm, we have a text field (<input type="text">) and a textarea (<textarea>) field.

3) In the TaskForm, we have a dropdown box with the options of all our Projects, a text input for the task name and a textarea field for the task description.

Let’s go to our controllers/root.py file. Let’s add these 4 actions there.

# these lines go before RootController class definition.
from ideas.model.project import Project, Task
from ideas.forms.project_form import project_form, task_form

# these lines go in the RootController class.
@expose('ideas.templates.new_project')
def new_project(self):
    """ Displays the project form. """
    return dict(project_form=project_form)

@expose('ideas.templates.new_task')
def new_task(self):
    """ Displays the task form. """
    return dict(task_form=task_form)

@expose()
def create_project(self, project_name, project_description):
    """
    Creates a project with the given name and description. 
    Redirects to the project itself.
    """
    project = Project(project_name=project_name, project_description=project_description)
    DBSession.add(project)
    DBSession.flush()
    return redirect("/show_project/%s" % project.id)

@expose()
def create_task(self, project_id, task_name, task_description, status):
    """
    Creates a task for the given project with the given information.
    Redirects back to the project.
    """
    task = Task(project_id=project_id, task_name=task_name, task_description=task_description, status=status)
    DBSession.add(task)
    DBSession.flush()
    return redirect("/show_project/%s" % project_id)

@expose('ideas.templates.show_project')
def show_project(self, project_id):
    """
    Shows the project with the given id and lists its tasks as well.
    Redirects to the new project page if the project does not exist. 
    """
    project = DBSession.query(Project).filter_by(id=project_id).first()
    if not project:
        return redirect("/new_project")
    tasks = DBSession.query(Task).filter_by(project_id=project_id).all()
    return dict(project=project, tasks=tasks)

Now you can create a project at the “http://127.0.0.1:8080/new_project” address. Actually, you can’t create a project yet. Because we haven’t set up the templates. Let’s set up the templates.

I want you to love TurboGears, understand how to use TurboGears and how to become fully productive with it. For example, I can develop a fully fledged application that would take 3 weeks with something else in 1-1.5 weeks with TurboGears. For that, I think a project management tool fits best here. It requires working with a database, creating forms, authentication and authorization and so on.

Now, go ahead and create a project called “ideas”. What we are going to do is a simple, a very simple project management tool. We are going to create a Project model, a Task model.

cd tg2env
source bin/activate
paster quickstart -m ideas
# answer yes to authentication/authorization question.

We will “cd” into the “ideas/ideas” directory and create a forms module and create a project_form.py file while in there.

cd ideas/ideas
mkdir forms
touch forms/__init__.py
touch forms/project_form.py

Next, we are going to create a “project” module in ideas/model/ folder.

cd model && touch project.py
# project.py
# -*- coding: utf-8 -*-
"""Post model module."""

from ideas.model import DBSession, DeclarativeBase
from sqlalchemy import Column, ForeignKey
from sqlalchemy.types import Integer, Unicode, UnicodeText

class Project(DeclarativeBase):
    __tablename__ = 'projects'

    id = Column(Integer, primary_key=True)
    project_name = Column(Unicode(255))
    project_description = Column(UnicodeText)

    @classmethod
    def get_projects_for_dropdown(self):
        projects = DBSession.query(Project).all()
        project_list = [(project.id, project.project_name) for project in projects]
        return project_list

class Task(DeclarativeBase):
    __tablename__ = 'tasks'

    id = Column(Integer, primary_key=True)
    project_id = Column(ForeignKey("projects.id"))
    task_name = Column(Unicode(255))
    task_description = Column(UnicodeText)
    status = Column(Unicode(6))

What have we done?
We have created two models, 1) Project and 2) Task. We have imported the necessary modules from both our project and SQLAlchemy in order to be able to setup our models correctly. The very next step is to create the tables we have declared in the models. Go and open ideas/model/__init__.py file. Go to the end of file. (Here is a “vim” trick for you. When you press ESC and then type “:$”, you will go to the end of file.) Add the following line there:

from ideas.model.project import Project, Task

Run the following commands:

# in the top level ideas/ project directory
python setup.py develop
paster setup-app development.ini

“paster setup-app development.ini” will create all our tables, including the authentication tables. It will use the database backend defined in the development.ini file, line: 52. By default, it is SQLite.

In the next post, we will create the forms. Until then, take care and check out ToscaWidgets.

I have been doing some work on benchmarking the many template systems under turbogears, currently the most visible are Genshi (default), Mako and now Jinja (which will be better integrated from 2.2 onwards), thus I made a simple benchmark to test the difference on speed with each template.

First I’m using TurboGears2 from git branch 215 plus my jinja tg2devtools branch, this in order to preview the jinja based quickstart template. For each templating system a fresh project is created with all defaults, each project is setup and run under paster and gunicorn for each test (this to compare development environment and a more production like environment).

The tests where done using apache bench (ab2) v2.3, with concurrency of 2 and 1000 tests, I tested 4 pages, the welcome (index), the environ page (where it renders a table from environment variables), the manager_permission_only page and the editor_user_only page, last 2 ones to test the template+sqlalchemy.

Finally those tests are not absolute and only represent performance on my machine but should give you a good idea of the performance differences between templating systems, the test machine is a AMD A6-3410MX CPU with 4 cores at 1.6ghz, 4GB RAM and 500gb 5400rpm hard disk, running under opensuse 12.1 with custom Linux kernel 3.3.0

Result data was processed with python using matplotlib to do the graphs.

Paster tests

As we can see here both mako and jinja outperform genshi by a good margin, in this test jinja is slightly faster. What is interesting however is that the welcome page Genshi is almost as fast a jinja, I tough this was a glitch or something but it works out consistently after several tests, seems to me that because the welcome page is relatively static genshi does a good job caching.

Gunicorn tests

Gunicorn tests are done by running gunicorn_paste, with 2 sync workers. The results:

Here we see it again, on the welcome page Genshi performs surprisingly on par with Jinja and Mako, but other tests really show, here Jinja is consistently faster than Mako.

Conclusions

As we can see Jinja is a good option for those looking for extra performance and are familiar already with Django templates, otherwise Mako is another great performance alternative under tg2.

Finally, I tried this test under pypy to see if I could squeeze even a bit more performance but the results where not consistent, the jit performance varies a lot, sometimes I get over 1000 request per second after jit warmup, but overall the results keep fluctuating, I can’t really say that running under pypy is a win for TG, although results seem to imply that it is a bit faster most of the time.

I have currently submitted a proposal for adding a set of jinja based templates to the tg2 quickstart command, with it the paster quickstart command will offer you the option to use jinja based templates, which means a full set of template files for jinja based development.

I tried to mimic as best as I could the way the Genshi templates are done, meaning that the rendering is exactly the same and that works more or less the same way, for example, using macros to define certain reusable parts of the page.

The probable timeframe for this patch to make it in mainline should be around v2.2 which is the current development branch, this patch however can work with older releases, you should have at least TG v2.1.5 (currently unreleased, under the 215 branch) for it to work as only on this version a patch for jinja template name resolution based on dotted notation has been integrated, the last means that from 2.1.5 onwards jinja will be able to load templates named after their import, such as “myproject.templates.index”, instead of the path based ones such as “index.html”.

Patches live under my jinja branch of tg2devtools for those who would like to test it earlier:

https://sourceforge.net/u/clsdaniel/tg2devtools/ci/2dd496a588d8705802e46d226a750a40bce691a4/tree/

Recently I decided to give a quick benchmark for curiosity to the going to be branches of TurboGears2.

I quickstarted a simple genshi based application (plain turbogears2 quckstart) and then I created a plain controller method without template, to avoid counting the template generation overhead.

The application has been installed in three virtual environments: one with TG2.1.4, one with the development branch which is going to be TG2.2 and one with the development branch which is going to be TG2.3

The following graph reports the resulting requests/second that my pc has been able to serve on each turbogears version.

I have to admit that I’m quite happy with the results, the grow is steady and TG2.3 seems to be three times faster than the current turbogears while still being backward compatible (The benchmark application has been quickstarted with TG2.1.4 and ran without issues on all the three environments)

Now that you have learnt what is TurboGears and how to install it, you want to get your hands dirty with it. This is really nice. I’m glad you’ve decided to do that. You won’t regret this decision. TurboGears is like that guitar you have always wanted to buy. Now that you have it, you can start to rock.

Let’s go into our virtual environment and quickstart a project.

cd ~/projects/tg2env
source bin/activate
paster quickstart -x -n -m example
cd example
python setup.py develop
paster serve development.ini

What have we done:

• Went into our virtual environment and activated it.
• Created a project called “example” with no sqlalchemy, no authentication and authorization and with mako templates.
• We have initiated the project and ran the server.

Let’s see how “http://127.0.0.1:8080/” looks like.

Now go ahead and play along with it. Discover TurboGears. You can ask anything via the comments.

In order to install TurboGears we need Python setuptools and in order to create an isolated environment we need virtualenv package.  Why an isolated environment? With a virtual environment we don’t need to think about our packages clashing.

Install requirements:

 sudo apt-get install build-essential python-dev python-setuptools python-virtualenv 

Those installation requirements are not specific to TurboGears. If you are working with Python anyway and tried to use any Python package management that means you already have these packages.

The standard installation

$ virtualenv --no-site-packages tg2env 
$ cd tg2env/ 
$ source bin/activate 
(tg2env)$ easy_install -i http://tg.gy/current/index/ tg.devtools 
(tg2env)$ deactivate 

You may get errors depending on your system and your system’s requirements. If you have not seen any errors, congratulations, you have installed TurboGears 2.1.4 (the current stable release as of this writing) and its dependencies successfully.

The pip way

You have tried to install TurboGears with the standard way however you are curious. How to install with pip as well? Fear not, young TGer. I will feed your hunger.

pip install -i http://tg.gy/current tg.devtools

Let’s create an example TurboGears project.

Hello everyone!

Recently I have seen a cloud service called DotCloud. It seemed nice and I wanted to check its Python support. I've looked at their documentation and saw their Django docs which basically means they support WSGI. I also wanted to see how much effort does it take to deploy on this service. I have jumped in, signed up and tried to deploy a very simple quickstarted TurboGears project to DotCloud. It went well.

Let's go ahead and deploy a quickstarted project.

Im Innern eines Waldorf Microwave II man sagt ja immer, man soll Synthesizer nicht nur nach den technischen Daten, sondern nach ihren inneren Werten beurteilen. Nun, wenn es danach ginge, könnte man meinen, dass der Waldorf Microwave II nicht viel zu bieten habe. Dass dem nicht so ist, erfahrt ihr hier demnächst in einem Review [...]