Sofeng's *Old* Blog: django

Showing posts with label django. Show all posts

Django Blog Project #9: Migrating Blogger posts with Beautiful Soup

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/07/django-blog-project-9-migrating-blogger/

You should be redirected in 2 seconds.

Last post, I talked about adding comments to my new sample blog application. This was about the last basic feature I needed to add before I started actually using it for real. Of course there are still a number of features I'd like to add, such as automatic syntax highlighting with Pygments, and incorporating django-tagging and some more intersting views, not to mention comment moderation. But I think those will have to wait-- I want to start using my new blog for real sometime.

So for the past few days, I've been working on my Beautiful Soup screen scraper script to copy all my Blogger posts over to my new Django blog. Initial results came quickly (it's pretty cool to see such a huge data dump after only a few lines of Beautiful Soup'ing) but the details (especially with the comments) kind of slowed me down. I've finally got everything copied over to my satisfaction. Below is the script I used to do it. Note, I realize it's not pretty-- just a one time use hack. But hopefully someone else doing the same thing might find it useful.

#!/usr/bin/env python

import datetime
import os
import re
import urllib2
from BeautifulSoup import BeautifulSoup
from myblogapp.models import Post, LegacyComment
from django.contrib.comments.models import FreeComment

URL = ''.join([
        'http://iwiwdsmi.blogspot.com/search?',
        'updated-min=2006-01-01T00%3A00%3A00-08%3A00&'
        'updated-max=2009-01-01T00%3A00%3A00-08%3A00&',
        'max-results=1000'
        ])
html = urllib2.urlopen(URL).read()
soup = BeautifulSoup(html)

for post in soup.html.body.findAll('div', {'class': 'post'}):
    print
    print '--------------------------------------------------------------'

    # save the post title and permalink
    h3 = post.find('h3', {'class': 'post-title'})
    post_href = h3.find('a')['href']
    post_title = h3.find('a').string
    post_slug = os.path.basename(post_href).rstrip('.html')
    print post_slug
    print post_href
    print post_title

    # save the post body
    div = post.find('div', {'class': 'post-body'})
    [toremove.extract() for toremove in div.findAll('script')]
    [toremove.extract() for toremove in div.findAll('span', {'id': 'showlink'})]
    [toremove.extract() for toremove in div.findAll('div', {'style': 'clear: both;'})]
    [toremove.parent.extract() for toremove in div.findAll(text='#fullpost{display:none;}')]
    post_body = ''.join([str(item)
                         for item in div.contents
                         ]).rstrip()
    post_body = re.sub(r"iwiwdsmi\.blogspot\.com/(\d{4}/\d{2}/[\w\-]+)\.html", 
                       r"www.saltycrane.com/blog/\1/", 
                       post_body)

    # count number of highlighted code sections 
    highlight = div.findAll('div', {'class': 'highlight'})
    if highlight:
        hl_count += len(highlight)
        hl_list.append(post_title)

    # save the timestamp
    a = post.find('a', {'class': 'timestamp-link'})
    try:
        post_timestamp = a.string
    except:
        match = re.search(r"\.com/(\d{4})/(\d{2})/", post_href)
        if match:
            year = match.group(1)
            month = match.group(2)
        post_timestamp = "%s/01/%s 11:11:11 AM" % (month, year)
    print post_timestamp

    # save the tags (this is ugly, i know)
    if 'error' in post_title.lower():
        post_tags = ['error']
    else:
        post_tags = []
    span = post.find('span', {'class': 'post-labels'})
    if span:
        a = span.findAll('a', {'rel': 'tag'})
    else:
        a = post.findAll('a', {'rel': 'tag'})
    post_tags = ' '.join([tag.string for tag in a] + post_tags)
    if not post_tags:
        post_tags = 'untagged'
    print post_tags

    # add Post object to new blog
    if True:
        p = Post()
        p.title = post_title
        p.body = post_body
        p.date_created = datetime.datetime.strptime(post_timestamp, "%m/%d/%Y %I:%M:%S %p")
        p.date_modified = p.date_created
        p.tags = post_tags
        p.slug = post_slug
        p.save()

    # check if there are comments
    a = post.find('a', {'class': 'comment-link'})
    if a:
        comm_string = a.string.strip()
    else:
        comm_string = "0"
    if comm_string[0] != "0":
        print
        print "COMMENTS:"

        # get the page with comments
        html_single = urllib2.urlopen(post_href).read()
        soup_single = BeautifulSoup(html_single)

        # get comments
        comments = soup_single.html.body.find('div', {'class': 'comments'})
        cauth_list = comments.findAll('dt')
        cbody_list = comments.findAll('dd', {'class': 'comment-body'})
        cdate_list = comments.findAll('span', {'class': 'comment-timestamp'})

        if not len(cauth_list)==len(cbody_list)==len(cdate_list):
            raise "didn't get all comment data"

        for auth, body, date in zip(cauth_list, cbody_list, cdate_list):
            
            # create comment in database
            lc = LegacyComment()
            lc.body = str(body.p)

            # find author
            lc.author = "Anonymous"
            auth_a = auth.findAll('a')[-1]
            auth_no_a = auth.contents[2]
            if auth_a.string:
                lc.author = auth_a.string
            elif auth_no_a:
                match = re.search(r"\s*([\w\s]*\w)\s+said", str(auth_no_a))
                if match:
                    lc.author = match.group(1)
            print lc.author

            # find website
            try:
                lc.website = auth_a['href']
            except KeyError:
                lc.website = ''
            print lc.website

            # other info
            lc.date_created = datetime.datetime.strptime(
                date.a.string.strip(), "%m/%d/%Y %I:%M %p")
            print lc.date_created
            lc.date_modified = lc.date_created
            lc.post_id = p.id
            lc.save()

I also made some changes to my Django blog code as I migrated my Blogger posts. The main addition was a LegacyComment model along with the associated views and templates. My Blogger comments consisted of HTML markup, but I didn't want to allow arbitrary HTML in my new comments for fear of cross site scripting. So I separated my legacy Blogger comments from my new Django site comments.

models.py

Here are my model changes. I added a LegacyComment class which contains pertinent comment attributes and a ForeignKey to the post that it belongs to. I also added a lc_count (for legacy comment count) field to the Post class which stores the number of comments for the post. It is updated by the save() method in the LegacyComment class every time a comment is saved. Hmmm, I just realized the count will be wrong if I ever edit these comments. Well, since these are legacy comments, hopefully I won't have to edit them.

~/src/django/myblogsite/myblogapp/models.py:

import re
from django.db import models

class Post(models.Model):
    title = models.CharField(maxlength=200)
    slug = models.SlugField(maxlength=100)
    date_created = models.DateTimeField() #auto_now_add=True)
    date_modified = models.DateTimeField()
    tags = models.CharField(maxlength=200)
    body = models.TextField()
    body_html = models.TextField(editable=False, blank=True)
    lc_count = models.IntegerField(default=0, editable=False)

    def get_tag_list(self):
        return re.split(" ", self.tags)

    def get_absolute_url(self):
        return "/blog/%d/%02d/%s/" % (self.date_created.year,
                                      self.date_created.month,
                                      self.slug)

    def __str__(self):
        return self.title

    class Meta:
        ordering = ["-date_created"]

    class Admin:
        pass

class LegacyComment(models.Model):
    author = models.CharField(maxlength=60)
    website = models.URLField(core=False)
    date_created = models.DateTimeField()
    date_modified = models.DateTimeField()
    body = models.TextField()
    post = models.ForeignKey(Post)

    def save(self):
        p = Post.objects.get(id=self.post.id)
        p.lc_count += 1
        p.save()
        super(LegacyComment, self).save()

    class Meta:
        ordering = ["date_created"]

    class Admin:
        pass

views.py

Here is an excerpt from my views.py file showing the changes:

~/src/django/myblogsite/myblogapp/views.py:

import re
from datetime import datetime
from django.shortcuts import render_to_response
from myblogsite.myblogapp.models import Post, LegacyComment

MONTH_NAMES = ('', 'January', 'Feburary', 'March', 'April', 'May', 'June', 'July',
               'August', 'September', 'October', 'November', 'December')
MAIN_TITLE = "Sofeng's Blog 0.0.7"

def frontpage(request):
    posts, pagedata = init()
    posts = posts[:5]
    pagedata.update({'post_list': posts,
                     'subtitle': '',})
    return render_to_response('listpage.html', pagedata)

def singlepost(request, year, month, slug2):
    posts, pagedata = init()
    post = posts.get(date_created__year=year,
                            date_created__month=int(month),
                            slug=slug2,)
    legacy_comments = LegacyComment.objects.filter(post=post.id)
    pagedata.update({'post': post,
                     'lc_list': legacy_comments,})
    return render_to_response('singlepost.html', pagedata)

Templates

In the list page template I used the truncatewords_html template filter to show a 50 word post summary on the list pages instead of the full post. I also added the legacy comment count with the Django free comment count to display the total number of comments.

Excerpt from ~/src/django/myblogsite/templates/listpage.html:

{% block main %}
  <br>
  {% for post in post_list %}
    <h4><a href="/blog/{{ post.date_created|date:"Y/m" }}/{{ post.slug }}/">
        {{ post.title }}</a>
    </h4>
    {{ post.body|truncatewords_html:"50" }}
    <a href="{{ post.get_absolute_url }}">Read more...</a><br>
    <br>
    <hr>
    <div class="post_footer">
      {% ifnotequal post.date_modified.date post.date_created.date %}
        Last modified: {{ post.date_modified.date }}<br>
      {% endifnotequal %}
      Date created: {{ post.date_created.date }}<br>
      Tags: 
      {% for tag in post.get_tag_list %}
        <a href="/blog/tag/{{ tag }}/">{{ tag }}</a>{% if not forloop.last %}, {% endif %}
      {% endfor %}
      <br>

      {% get_free_comment_count for myblogapp.post post.id as comment_count %}
      <a href="{{ post.get_absolute_url }}#comments">
        {{ comment_count|add:post.lc_count }} 
        Comment{{ comment_count|add:post.lc_count|pluralize}}</a>

    </div>
    <br>
  {% endfor %}
{% endblock %}

In the single post template, I added the display of the Legacy comments in addition to the Django free comments.

Excerpt from ~/src/django/myblogsite/templates/singlepost.html:

  <a name="comments"></a>
  {% if lc_list %}
    <h4>{{ lc_list|length }} Legacy Comment{{lc_list|length|pluralize}}</h4>
  {% endif %}
  {% for legacy_comment in lc_list %}
    <br>
    <a name="lc{{ legacy_comment.id }}" href="#lc{{ legacy_comment.id }}">
      #{{ forloop.counter }}</a>
    {% if legacy_comment.website %}
      <a href="{{ legacy_comment.website }}">
        <b>{{ legacy_comment.author|escape }}</b></a> 
    {% else %}
      <b>{{ legacy_comment.author|escape }}</b>
    {% endif %}
    commented,
    on {{ legacy_comment.date_created|date:"F j, Y" }} 
    at {{ legacy_comment.date_created|date:"P" }}:
    {{ legacy_comment.body }}
  {% endfor %}
  <br>

That's it. Hopefully, I can start using my new blog soon. Please browse around on the new Django site and let me know if you run across any problems. When everything looks to be OK, I'll start posting only on my new Django site.

Here is a snapshot screenshot of version 0.0.8:

The live site can be viewed at: http://saltycrane.com/blog

Django Blog Project#8: Adding basic comment functionality with Django Free Comments

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/07/django-blog-project8-adding-basic/

You should be redirected in 2 seconds.

This post describes how I added basic commenting functionality using Django's Free Comments. Note, this built-in functionality does not support comment moderation (used to counteract spam) or other nice features. I tried to use David Bennett's comment_utils, but I couldn't get it to work. So for now, I'm just going to use Django Free Comments and hopefully add in moderation and other features later.

I basically followed the instructions on the wiki: Using Django's Free Comments. See there for more information. It was pretty easy to set up.

settings.py

In my settings.py file, I added django.contrib.comments to the list of installed apps.

Excerpt from ~/src/django/myblogsite/settings.py:

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.admin',
    'django.contrib.comments',
    'iwiwdsmi.myblogapp',
)

urls.py

I modified my urls.py to use the comment module's URLConf.

~/src/django/myblogsite/urls.py:

from django.conf.urls.defaults import *
from django.contrib.comments.models import FreeComment
from iwiwdsmi.myblogapp.views import *
from iwiwdsmi.feeds import *

feeds = {
    'latest': LatestPosts,
}

urlpatterns = patterns(
    '',
    (r'^site_media/(?P<path>.*)$', 'django.views.static.serve', {'document_root': '/home/sofeng/src/django/myblogsite/media'}),

    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^feeds/(?P<url>.*)/$', 'django.contrib.syndication.views.feed', 
     {'feed_dict': feeds}),
    (r'^comments/', include('django.contrib.comments.urls.comments')),
    
    (r'^myview1/$', myview1),
    (r'^blog/$', frontpage),
    (r'^blog/(\d{4})/(\d{2})/([\w\-]+)/$', singlepost),
    (r'^blog/(\d{4})/$', yearview),
    (r'^blog/(\d{4})/(\d{2})/$', monthview),
    (r'^blog/tag/([\w\-]+)/$', tagview),
)

python manage.py syncdb

To install the comments model, I ran python manage.py syncdb:

$ cd ~/src/django/myblogsite
$ python manage.py syncdb

List page template

Then I modified my list page template to display the number of comments for each post:

~/src/django/myblogsite/templates/listpage.html:

{% extends "base.html" %}

{% load comments %}

{% block title %}
  {{ main_title }}
  {% if subtitle %}:{% endif %}
  {{ subtitle }}
{% endblock %}

{% block header1 %}
  {% if subtitle %}
    <a href="/blog/">{{ main_title }}</a>
  {% else %}
    {{ main_title }}
  {% endif %}
{% endblock %}

{% block header2 %}
  {{ subtitle }}
{% endblock %}

{% block main %}
  {% for post in post_list %}
    <h3><a href="/blog/{{ post.date_created|date:"Y/m" }}/{{ post.slug }}/">
        {{ post.title }}</a>
    </h3>
    {{ post.body }}
    <hr>
    <div class="post_footer">
      {% ifnotequal post.date_modified.date post.date_created.date %}
        Last modified: {{ post.date_modified.date }}<br>
      {% endifnotequal %}
      Date created: {{ post.date_created.date }}<br>
      Tags: 
      {% for tag in post.get_tag_list %}
        <a href="/blog/tag/{{ tag }}/">{{ tag }}</a>{% if not forloop.last %}, {% endif %}
      {% endfor %}
      <br>
      {% get_free_comment_count for myblogapp.post post.id as comment_count %}
      <a href="{{ post.get_absolute_url }}">{{ comment_count }} Comment{{ comment_count|pluralize}}</a>
    </div>
    <br>
  {% endfor %}
{% endblock %}

Oh, I forgot to mention, I implemented a get_absolute_url method in my Post model. This is the preferred way to specify the url to the detail view of my Post object. The code for the method is shown below:

~/src/django/myblogsite/myblogapp/models.py:

import re
from django.db import models

class Post(models.Model):
    title = models.CharField(maxlength=200)
    slug = models.SlugField(maxlength=100)
    date_created = models.DateTimeField() #auto_now_add=True)
    date_modified = models.DateTimeField(auto_now=True)
    tags = models.CharField(maxlength=200)
    body = models.TextField()

    def get_tag_list(self):
        return re.split(" ", self.tags)

    def get_absolute_url(self):
        return "/blog/%d/%02d/%s/" % (self.date_created.year,
                                      self.date_created.month,
                                      self.slug)

    def __str__(self):
        return self.title

    class Meta:
        ordering = ["-date_created"]

    class Admin:
        pass

Detail page template

Then I modified my single post, or detail page, template:

~/src/django/myblogsite/templates/singlepost.html:

{% extends "base.html" %}

{% load comments %}

{% block title %}
  {{ main_title }}: {{ post.title }}
{% endblock %}

{% block header1 %}
  <a href="/blog/">{{ main_title }}</a>
{% endblock %}

{% block main %}
  <h3>{{ post.title }}</h3>
  {{ post.body }}
  <hr>
  <div class="post_footer">
    {% ifnotequal post.date_modified.date post.date_created.date %}
      Last modified: {{ post.date_modified.date }}<br>
    {% endifnotequal %}
    Date created: {{ post.date_created.date }}<br>
    Tags: 
    {% for tag in post.get_tag_list %}
      <a href="/blog/tag/{{ tag }}/">{{ tag }}</a>{% if not forloop.last %}, {% endif %}
    {% endfor %}
  </div>
  <br>
  {% get_free_comment_list for myblogapp.post post.id as comment_list %}
  {% get_free_comment_count for myblogapp.post post.id as comment_count %}
  {% if comment_list %}
    <h4>{{ comment_count }} Comment{{ comment_count|pluralize}}</h4>
  {% endif %}
  {% for comment in comment_list %}
    <a href="#c{{ comment.id }}">#{{ forloop.counter }}</a>
    <b>{{ comment.person_name|escape }}</b> commented, 
      on {{ comment.submit_date|date:"F j, Y" }} at {{ comment.submit_date|date:"P" }}:
    {{ comment.comment|escape|urlizetrunc:40|linebreaks }}
  {% endfor %}
  <br>
  <h4>Post a comment</h4>
  {% free_comment_form for myblogapp.post post.id %}

{% endblock %}

Create templates for comment preview and comment posted

At this point, I can view these template changes on my blog list view or detail view. However, if I try to add a comment, I will get a TemplateDoesNotExist exception. I need two new templates-- one for the comment preview and one for the page just after the comment is posted. I just copied these from the wiki. Probably, I should dress these up a little to match my site. For now, I'll use the generic ones. I put them in a new directory called comments in my templates directory.

~/src/django/myblogsite/templates/comments/free_preview.html:

<h1>Preview your comment</h1>

<form action="/comments/postfree/" method="post">
    {% if comment_form.has_errors %}
        <p><strong style="color: red;">Please correct the following errors.</strong></p>
    {% else %}
        <div class="comment">
        {{ comment.comment|escape|urlizetrunc:"40"|linebreaks }}
        <p class="date small">Posted by <strong>{{ comment.person_name|escape }}</strong></p>
        </div>

        <p><input type="submit" name="post" value="Post public comment" /></p>

        <h1>Or edit it again</h1>
    {% endif %}

    {% if comment_form.person_name.errors %}
        {{ comment_form.person_name.html_error_list }}
    {% endif %}

    <p><label for="id_person_name">Your name:</label> {{ comment_form.person_name }}</p>

    {% if comment_form.comment.errors %}
        {{ comment_form.comment.html_error_list }}
    {% endif %}

    <p>
        <label for="id_comment">Comment:</label>
        <br />
        {{ comment_form.comment }}
    </p>
    
    <input type="hidden" name="options" value="{{ options }}" />
    <input type="hidden" name="target" value="{{ target }}" />
    <input type="hidden" name="gonzo" value="{{ hash }}" />

    <p>
        <input type="submit" name="preview" value="Preview revised comment" />
    </p>
</form>

~/src/django/myblogsite/templates/comments/posted.html:

<h1>Comment posted successfully</h1>

<p>Thanks for contributing.</p>

{% if object %}
    <ul>
        <li><a href="{{ object.get_absolute_url }}">View your comment</a></li>
    </ul>
{% endif %}

Add some comments

I'm done. I added some comments and saw them show up on my page.

Upload, update, restart server

I uploaded to webfaction, updated my Mercurial repository, and restarted the Apache server. Everything's good.

Here is a snapshot screenshot of a detail page with some comments:

The live site can be viewed at: http://saltycrane.com/blog

Django Blog Project #7: Adding a simple Atom feed

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/06/django-blog-project-7-adding-simple/

You should be redirected in 2 seconds.

Setting up a basic Atom feed for my new blog was not too difficult. I followed the instructions in the Django documentation and created a basic Atom feed for my 10 latest posts. There were a couple of things I couldn't figure out: 1) how to create a feed for a specific tag, and 2) how to redirect my feed to FeedBurner. I know these things are not that hard, but I want to get my new blog working as soon as possible so I can start using it. If anyone knows how to do these things, I'd be grateful for your feedback. Regarding item 1, I plan to replace my tagging functionality hack with the django-tagging application so maybe I will wait until then to add tag-specific feeds. Regarding item 2, I read about using .htaccess to do redirects, but I couldn't figure out how to do a feed redirect at Webfaction.

Anyways, here are the steps I took to create a basic feed of my latests posts. Refer to the Django syndication feed framework documentation for more details.

Modify the URLConf
Per the documentation, I modified my ~/src/django/myblogsite/urls.py:

from django.conf.urls.defaults import *
from iwiwdsmi.myblogapp.views import *
from iwiwdsmi.feeds import *

feeds = {
    'latest': LatestPosts,
}

urlpatterns = patterns(
    '',
    (r'^site_media/(?P<path>.*)$', 'django.views.static.serve', {'document_root': '/home/sofeng/src/django/myblogsite/media'}),

    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^feeds/(?P<url>.*)/$', 'django.contrib.syndication.views.feed', 
     {'feed_dict': feeds}),    
    
    (r'^blog/$', frontpage),
    (r'^blog/(\d{4,4})/(\d{2,2})/([\w\-]+)/$', singlepost),
    (r'^blog/(\d{4,4})/$', yearview),
    (r'^blog/(\d{4,4})/(\d{2,2})/$', monthview),
    (r'^blog/tag/([\w\-]+)/$', tagview),
)

Create a feeds.py file

I created the following file at ~/src/django/myblogsite/feeds.py. Note, I factored out my main title, "Sofeng's Blog 0.0.6", into the variable MAIN_TITLE in my views.py file and used it here. The class attributes, title and description are strings describing the feed.

from django.contrib.syndication.feeds import Feed
from django.utils.feedgenerator import Atom1Feed
from iwiwdsmi.myblogapp.models import Post
from iwiwdsmi.myblogapp.views import MAIN_TITLE

class LatestPosts(Feed):
    title = MAIN_TITLE
    link = "/blog/"
    description = MAIN_TITLE + ": Latest posts"
    feed_type = Atom1Feed
    
    def items(self):
        return Post.objects.order_by('-date_created')[:10]

Create feed templates

I created two feed templates which are used to display the title and body for each post. They each only contain one line.

~/src/django/myblogsite/templates/feeds/latest_title.html:

{{ obj.title }}

~/src/django/myblogsite/templates/feeds/latest_description.html:

{{ obj.body }}

Display a link to the new feed

I added a link to my new feed in my base.html template.

Excerpt from ~/src/django/myblogsite/templates/base.html:

        <h4>FEEDS</h4>
          <a href="/feeds/latest/" rel="alternate" type="application/rss+xml"><img alt="" style="vertical-align:middle;border:0" src="http://www.feedburner.com/fb/images/pub/feed-icon16x16.png"/></a>
        <a href="/feeds/latest/">Subscribe to Atom feed</a><br>

Change site name

When I first created my Django project, I installed the django.contrib.sites app (by default), but I did not change my site name, so my feed used the site example.com instead of my real site. Here is how I changed my site name:

I went to the admin page at http://127.0.0.1:8000/admin/
I clicked on "Sites", then on "example.com".
I changed "example.com" to "saltycrane.com" and clicked the "Save" button.

Deploy and Test

I pushed my changes to the Webfaction server, updated my repository, and restarted the Apache server. Then I pointed Firefox at http://saltycrane.com/blog/ and clicked on my new "Subscribe to Atom feed" link. I subscribed to the feed using "Google" and chose "Add to Google Reader" and saw my new feed in Google Reader.
Here is a screenshot of my feed in my Google Reader:

Extra links on redirection for my reference:
Webfaction knowledgebase article on redirection
Blog article on redirecting feeds to FeedBurner

Related posts:
   Django Blog Project #1: Creating a basic blog
   Django Blog Project #2: Deploying at Webfaction
   Django Blog Project #3: Using CSS and Template Inheritance
   Django Blog Project #4: Adding post metadata
   Django Blog Project #5: YUI CSS and serving static media
   Django Blog Project #6: Creating standard blog views
   Django Blog Project #8: Adding basic comment functionality with Django Free Comments

[Read the full post...]

Django Blog Project #6: Creating standard blog views

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/06/django-blog-project-6-creating-standard/

You should be redirected in 2 seconds.

My last django post actually got a little bit of publicity-- a mention on Python reddit, and a mention in the Yahoo User Interface blog In the Wild for June 20 article. So I feel a little pressure to produce intelligent, engaging content so as not to disappoint the hordes of new readers I must now have. Unfortunately, the topic of this post is pretty dry. Goodbye, new readers. I'll try to find you another time.

The topic of this post is creating standard blog views. So far, I've created a simple blog application that allows me to log in to an Admin interface and enter blog posts, I added some post metadata such as date and tags, and, last time, I added YUI stylesheets to create a cross browser layout solution. However, until now, my trying-so-hard-to-be-a-real website only had a single front page view with a list of all the blog posts. I needed to add other views, such as a single post view, an archive view, and a tag view. Here are the changes I made to implement these views.

Warning: I am a newbie to Django and web development. I've created this blog series to document my journey in creating a blog application using Django. It does not necessarily represent Django best practices. To the extent of my knowledge, I try to follow Django conventions and philosophies, but I know I must still have a number of violations, particularly in this latest revision. Hopefully, as I learn better ways of doing things, I can post corrections. If you have corrections, please leave a comment.

For a much better example of a Django blog application, see Adam Gomaa's Django blog site source code repository. Though I don't understand everything in there, I've referenced it a number of times while creating my blog site. Update 2008-06-29: I just discovered that James Bennett, a Django contributer, also has blog application source code available online. Seeing as he has written a book on Django, this <sarcasm>might</sarcasm> be a good place to look as well.

models.py

In my models.py file, I added a slug field to store the last part of the url and a method called get_tag_list to return the tags for the post as a list.

~/src/django/myblogsite/myblogapp/models.py:

import re
from django.db import models

class Post(models.Model):
    title = models.CharField(maxlength=200)
    slug = models.SlugField(maxlength=100)
    date_created = models.DateTimeField()
    date_modified = models.DateTimeField(auto_now=True)
    tags = models.CharField(maxlength=200)
    body = models.TextField()

    def get_tag_list(self):
        return re.split(" ", self.tags)

    def __str__(self):
        return self.title

    class Meta:
        ordering = ["-date_created"]

    class Admin:
        pass

views.py

In my views.py file, to the frontpage view, I added a single post view, an archive view by year, an archive view by month, and a tag view. I also added a couple functions to create an archive index and a tag index.

~/src/django/myblogsite/myblogapp/views.py:

import re
from datetime import datetime
from django.shortcuts import render_to_response
from myblogsite.myblogapp.models import Post

MONTH_NAMES = ('', 'January', 'Feburary', 'March', 'April', 'May', 'June', 'July',
               'August', 'September', 'October', 'November', 'December')

def frontpage(request):
    posts, pagedata = init()
    pagedata.update({'subtitle': '',})
    return render_to_response('listpage.html', pagedata)

def singlepost(request, year, month, slug2):
    posts, pagedata = init()
    post = posts.get(date_created__year=year,
                            date_created__month=int(month),
                            slug=slug2,)
    pagedata.update({'post': post})
    return render_to_response('singlepost.html', pagedata)
    
def yearview(request, year):
    posts, pagedata = init()
    posts = posts.filter(date_created__year=year)
    pagedata.update({'post_list': posts,
                     'subtitle': 'Posts for %s' % year})
    return render_to_response('listpage.html', pagedata)

def monthview(request, year, month):
    posts, pagedata = init()
    posts = posts.filter(date_created__year=year)
    posts = posts.filter(date_created__month=int(month))
    pagedata.update({'post_list': posts,
                     'subtitle': 'Posts for %s %s' % (MONTH_NAMES[int(month)], year),})
    return render_to_response('listpage.html', pagedata)

def tagview(request, tag):
    allposts, pagedata = init()
    posts = []
    for post in allposts:
        tags = re.split(' ', post.tags)
        if tag in tags:
            posts.append(post)
    pagedata.update({'post_list': posts,
                     'subtitle': "Posts tagged '%s'" % tag,})
    return render_to_response('listpage.html', pagedata)

def init():
    posts = Post.objects.all()
    tag_data = create_tag_data(posts)
    archive_data = create_archive_data(posts)
    pagedata = {'version': '0.0.5',
                'post_list': posts,
                'tag_counts': tag_data,
                'archive_counts': archive_data,}
    return posts, pagedata

def create_archive_data(posts):
    archive_data = []
    count = {}
    mcount = {}
    for post in posts:
        year = post.date_created.year
        month = post.date_created.month
        if year not in count:
            count[year] = 1
            mcount[year] = {}
        else:
            count[year] += 1
        if month not in mcount[year]:
            mcount[year][month] = 1
        else:
            mcount[year][month] += 1
    for year in sorted(count.iterkeys(), reverse=True):
        archive_data.append({'isyear': True,
                             'year': year, 
                             'count': count[year],})
        for month in sorted(mcount[year].iterkeys(), reverse=True):
            archive_data.append({'isyear': False,
                                 'yearmonth': '%d/%02d' % (year, month),
                                 'monthname': MONTH_NAMES[month], 
                                 'count': mcount[year][month],})
    return archive_data

def create_tag_data(posts):
    tag_data = []
    count = {}
    for post in posts:
        tags = re.split(" ", post.tags)
        for tag in tags:
            if tag not in count:
                count[tag] = 1
            else:
                count[tag] += 1
    for tag, count in sorted(count.iteritems(), key=lambda(k, v): (v, k), reverse=True):
        tag_data.append({'tag': tag,
                         'count': count,})
    return tag_data

Templates
I made the following template changes:

I modified the sidebar in base.html to display an archive index and a tag index.
I replaced frontpage.html with a more generic listpage.html template used for displaying the frontpage, archives, and tag views.
I added a singlepost.html template for displaying a single post view.
In all the templates, I created links to navigate among the different views.

~/src/django/myblogsite/templates/base.html:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" 
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  <link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.5.2/build/reset-fonts-grids/reset-fonts-grids.css">
  <link rel="stylesheet" type="text/css" href="/site_media/css/mystyle.css">
  <title>{% block title %}Sofeng's Blog {{ version }}{% endblock %}</title>
</head>

<body>
  <div id="doc" class="yui-t4">

    <div id="hd">
      <h1>
        <img align="center" src="/site_media/image/crane.jpg">
        {% block header1 %}Sofeng's Blog {{ version }}{% endblock %}
      </h1>
      <h2>{% block header2 %}{% endblock %}</h2>
    </div>

    <div id="bd">
      <div id="yui-main">
        <div class="yui-b">
          {% block main %}{% endblock %}
        </div>
      </div>
    
      <div class="yui-b" id="sidebar">
        {% block sidebar %}
        <h4>ABOUT</h4>
        <p>This is my new blog created using <a href="http://www.djangoproject.com">Django</a>,
          a <a href="http://www.python.org">Python</a> web framework. This site is under 
          construction. My current blog is located at: 
          <a href="http://iwiwdsmi.blogspot.com">http://iwiwdsmi.blogspot.com</a>.
        </p>
        <br>
        <h4>TAGS</h4>
        {% for line in tag_counts %}
            <a href="/blog/tag/{{ line.tag }}/">{{ line.tag }}</a> ({{ line.count }})<br>
        {% endfor %}
        <br>
        <h4>ARCHIVE</h4>
        {% for line in archive_counts %}
          {% if line.isyear %}
            {% if not forloop.first %}
              <br>
            {% endif %}
              <a href="/blog/{{ line.year }}/">{{ line.year }}</a> ({{ line.count }})<br>
          {% else %}
                <a href="/blog/{{ line.yearmonth }}/">{{ line.monthname }}</a> ({{ line.count }})<br>
          {% endif %}
        {% endfor %}
        <br>
        {% endblock %}
      </div>
    </div>

    <div id="ft">
      Created with <a href="http://www.djangoproject.com/">Django</a>.<br>
      Hosted by <a href="http://www.webfaction.com/">Webfaction</a>.
    </div>
  </div>

</body>
</html>

~/src/django/myblogsite/templates/listpage.html:

{% extends "base.html" %}

{% block title %}
  Sofeng's Blog {{ version }}
  {% if subtitle %}:{% endif %}
  {{ subtitle }}
{% endblock %}

{% block header1 %}
  {% if subtitle %}
    <a href="/blog/">Sofeng's Blog {{ version }}</a>
  {% else %}
    Sofeng's Blog {{ version }}
  {% endif %}
{% endblock %}

{% block header2 %}
  {{ subtitle }}
{% endblock %}

{% block main %}
  {% for post in post_list %}
    <h3><a href="/blog/{{ post.date_created|date:"Y/m" }}/{{ post.slug }}/">
        {{ post.title }}</a>
    </h3>
    {{ post.body }}
    <hr>
    <div class="post_footer">
      {% ifnotequal post.date_modified.date post.date_created.date %}
        Last modified: {{ post.date_modified.date }}<br>
      {% endifnotequal %}
      Date created: {{ post.date_created.date }}<br>
      Tags: 
      {% for tag in post.get_tag_list %}
        <a href="/blog/tag/{{ tag }}/">{{ tag }}</a>{% if not forloop.last %}, {% endif %}
      {% endfor %}
    </div>
    <br>
  {% endfor %}
{% endblock %}

~/src/django/myblogsite/templates/singlepost.html:

{% extends "base.html" %}

{% block title %}
  Sofeng's Blog {{ version }}: {{ post.title }}
{% endblock %}

{% block header1 %}
  <a href="/blog/">Sofeng's Blog {{ version }}</a>
{% endblock %}

{# to fix IE #}
{% block header2 %} {% endblock %}

{% block main %}
    <h3>{{ post.title }}</h3>
    {{ post.body }}
    <hr>
    <div class="post_footer">
      {% ifnotequal post.date_modified.date post.date_created.date %}
        Last modified: {{ post.date_modified.date }}<br>
      {% endifnotequal %}
      Date created: {{ post.date_created.date }}<br>
      Tags: 
      {% for tag in post.get_tag_list %}
        <a href="/blog/tag/{{ tag }}/">{{ tag }}</a>{% if not forloop.last %}, {% endif %}
      {% endfor %}
    </div>
    <br>
{% endblock %}

urls.py

As the last step in my bottom-up approach, I modified my urls.py and urls_webfaction.py.

~/src/django/myblogsite/urls_webfaction.py:

from django.conf.urls.defaults import *
from myblogsite.myblogapp.views import *

urlpatterns = patterns(
    '',
    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^myview1/$', myview1),
    (r'^blog/$', frontpage),
    (r'^blog/(\d{4,4})/(\d{2,2})/([\w\-]+)/$', singlepost),
    (r'^blog/(\d{4,4})/$', yearview),
    (r'^blog/(\d{4,4})/(\d{2,2})/$', monthview),
    (r'^blog/tag/([\w\-]+)/$', tagview),
)

Finish

I uploaded, updated, and restarted the Apache server as usual.

Here is a snapshot screenshot of the site:

The live site is located at: http://saltycrane.com/blog/

Related posts:
   Django Blog Project #1: Creating a basic blog
   Django Blog Project #2: Deploying at Webfaction
   Django Blog Project #3: Using CSS and Template Inheritance
   Django Blog Project #4: Adding post metadata
   Django Blog Project #5: YUI CSS and serving static media
   Django Blog Project #7: Adding a simple Atom feed
   Django Blog Project #8: Adding basic comment functionality

[Read the full post...]

Django Blog Project #5: YUI CSS and serving static media

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/06/django-blog-project-5-yui-css-and/

You should be redirected in 2 seconds.

I wrote about adding some CSS to my new blog in my post #3. However, I got the CSS layout code from a book that was a couple of years old (CSS Cookbook, 2006). I created a 2 column fixed width layout, but it was not centered. It turns out creating a robust, cross-browser solution to centering a page layout is not trivial, even in 2008.

I happened to notice that the Django Book used YUI style sheets for styling its online book. The Yahoo User Interface Library (YUI) consists mostly of a Javascript Library, but it also includes "several core CSS resources". Doing a search among Django blogs using django blog search revealed that YUI was a pretty popular choice for CSS among Django users. Watching the 42 minute YUI CSS introductory video gave me a better understanding of the current state of CSS and browser compatability and how to use YUI to easily create consistent layouts across a variety of browsers.

So in this post I'll touch on how I used the YUI CSS library to restyle my slowly-getting-to-be-a-fledgling site. I'll also note how I setup Django and Webfaction to serve the static CSS media files. (What, I need a special setup to use external CSS stylesheets? you ask. I know, it suprised me too.) Finally, I made some small updates to my model to use more reasonable model attribute field types.

Setup YUI-based template

In my base.html template file, I removed all the CSS in between the <style></style> tags and replaced it with a link to the YUI concatenated, minified reset+fonts+grids aggregate file. I also added a link to my to-be-described-later local CSS stylesheet, mystyle.css. In the body section, I added a number of div blocks to create the required structure for the YUI library. The following code creates a fixed 750 pixel total width layout with a 180 pixel side column on the right and a full width header and footer. Not coming from a web background, I had thought this type of thing would be easier. I guess this complexity contributes to the demand for tools such as Dreamweaver.

~/src/django/myblogsite/templates/base.html:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" 
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  <link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.5.2/build/reset-fonts-grids/reset-fonts-grids.css">
  <link rel="stylesheet" type="text/css" href="/site_media/css/mystyle.css">
  <title>{% block title %}Sofeng's Blog Version {{ VERSION }}{% endblock %}</title>
</head>

<body>
  <div id="doc" class="yui-t4">
    <div id="hd">
      <h1>Sofeng's Blog Version {{ VERSION }}</h1>
    </div>
    <div id="bd">
      <div id="yui-main">
        <div class="yui-b">
          {% block main %}{% endblock %}
        </div>
      </div>
      <div class="yui-b" id="sidebar">
        {% block sidebar %}
        <h3>ABOUT</h3>
        <p>This is my new blog created using <a href="http://www.djangoproject.com">Django</a>,
          a <a href="http://www.python.org">Python</a> web framework. This site is under 
          construction. My current blog is located at: 
          <a href="http://iwiwdsmi.blogspot.com">http://iwiwdsmi.blogspot.com</a>.
        </p>
        {% endblock %}
      </div>
    </div>
    <div id="ft">
      A <a href="http://www.djangoproject.com/">Django</a> site.
    </div>
  </div>
</body>
</html>

Create local mystyle.css stylesheet

mystyle.css contains the CSS code to customize the style (fonts, colors, spacing, etc.) of my site. Currently it is just a copy of YUI's Base library with a few minor modifications. (YUI's Reset library removes all browser specific styling (located in browser.css or similar). Then the YUI Base library adds back the common element styles that we are familiar with. This provides consistency across browsers.) I saved this file at ~/src/django/myblogsite/media/css/mystyle.css.

Serve static CSS media files (development server)

Django doesn't serve static media files such as CSS, JavaScript, and images by default. The reason is that Django is meant to create dynamically generated webpages and serving static media is best left to a tool designed for this task such as Apache. However, during development, it is convenient to have the Django development server (python manage.py runserver) serve the static media files. To do this, I used these instructions from Django. This amounted to adding a new line to my URLConf (urls.py).

~/src/django/myblogsite/urls.py:

from django.conf.urls.defaults import *
from myblogsite.myblogapp.views import *

urlpatterns = patterns(
    '',
    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^myview1/$', myview1),
    (r'^blog/$', frontpage),
    (r'^site_media/(?P.*)$', 'django.views.static.serve', {'document_root': '/home/sofeng/src/django/myblogsite/media'}),
)

Serve static CSS media files (Webfaction server)

To serve my static CSS file at Webfaction, I followed these Webfaction knowledgebase instructions:

I went to the Webfaction Control Panel
Under the ">Domains/websites" menu, I selected "Applications"
I selected the icon with the plus to add a new App.
I filled in the following fields:
"Name": myblogsite_media "App type": Symbolic link "Extra info": /home/sofeng/webapps/django/myblogsite/media
Under the ">Domains/websites" menu, I selected "Websites"
I clicked on the pencil icon to edit my website
Under "Site apps", I selected the icon with the plus to add a new site app.
I selected "myblogsite_media" as the App and entered /site_media as the "URL path".

In my settings_webfaction.py file I updated the MEDIA_ROOT and MEDIA_URL variables. (Did I forget to mention I created a separate settings file for Webfaction. I point to the settings_webfaction.py file in my ~/webapps/django/apache2/conf/httpd.conf.) I also created a new file, urls_webfaction.py that doesn't have the above modification and I point to this URLConf in my settings_webfaction.py. I know there has got to be a better way to do this, but my tired brain hasn't figured it out yet.

In ~/src/django/myblogsite/settings_webfaction.py, I modified the following lines:

MEDIA_ROOT = '/home/sofeng/webapps/django/myblogsite/media/'
MEDIA_URL = '/site_media/'
ROOT_URLCONF = 'myblogsite.urls_webfaction'

Upload project and deploy
On my local machine:

$ pushwebf

On Webfaction machine:

$ hg update -C
$ ~/webapps/django/apache2/bin/restart

Here is a snapshot screenshot of the site:

The live site can be viewed at: http://saltycrane.com/blog/

Related posts:
   Django Blog Project #1: Creating a basic blog
   Django Blog Project #2: Deploying at Webfaction
   Django Blog Project #3: Using CSS and Template Inheritance
   Django Blog Project #4: Adding post metadata
   Django Blog Project #6: Creating standard blog views

[Read the full post...]

Django Blog Project #4: Adding post metadata

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/06/django-blog-project-4-adding-post/

You should be redirected in 2 seconds.

I've created a basic blog using SQLite as a backend, deployed it at a shared hosting provider, Webfaction, and added a little CSS for style. Now what? Well, while I say I've created a basic blog, I should say it's a not-quite-yet-basic blog. All it has is some text, which I call Posts, separated by <hr> tags. The next step is to add some post metadata, such as a title, date of creation and tags.

Modify the Post model

To create this new metadata, I first modified the Post model in ~/src/django/myblogsite/myblogapp/models.py:

from django.db import models

class Post(models.Model):
    title = models.CharField(maxlength=500)
    date_created = models.DateField()
    tags = models.CharField(maxlength=200)
    body = models.TextField()

    def __str__(self):
        return self.title

    class Meta:
        ordering = ["-id"]

    class Admin:
        pass

I added new attributes: title, date_created, and tags. The __str__ method is used to identify a Post instance by its title. This is useful when working in the Administration page. The class Meta: is used to reverse sort the Posts by "id".

Correction 7/6/2008: For the Post's body field, I previously used the line: body = models.CharField(maxlength=999999). However, thanks to Myles's comment below, I've changed this to use the more appropriate TextField.

python manage.py syncdb

Whoops, running python manage.py syncdb doesn't actually sync the model with the database. It is only used to initially create the database tables. To add the new columns to my table, I would have to enter the SQL commands directly. Though it is not super difficult, and would be good experience since all those hiring managers seem to want SQL experience, I took the lazy route and just replaced my database. (Not like I had much in there anyways.) For your reference, the section Making Changes to a Database Schema in Chapter 5 of The Django Book describes how to solve my problem without replacing my database.

$ cd ~/src/django/myblogsite
$ rm mydatabase.sqlite3 
$ python manage.py syncdb 
Creating table auth_message
Creating table auth_group
Creating table auth_user
Creating table auth_permission
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table django_admin_log
Creating table myblogapp_post

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (Leave blank to use sofeng'): sofeng
E-mail address: my@email.address
Password: 
Password (again): 
Superuser created successfully.
Installing index for auth.Message model
Installing index for auth.Permission model
Installing index for admin.LogEntry model
Loading 'initial_data' fixtures...
No fixtures found.

Oh yeah, I forgot gotta recreate my superuser.

Modify views.py slightly

In versions 0.0.1 and 0.0.2 of my new blog, I passed a list of post bodies to my template. But actually, I could have just passed a list of the Post objects and accessed the body attribute of each post in the template. This saves a line of unecessary code, and provides accessiblity to all the attributes of the Post object. (I also factored out my blog version number from the base.html template.)

~/src/django/myblogsite/myblogapp/views.py:

from django.shortcuts import render_to_response
from myblogsite.myblogapp.models import Post

def frontpage(request):
    posts = Post.objects.all()
    return render_to_response('frontpage.html', 
                              {'post_list': posts,
                               'VERSION': '0.0.3'})

Correction 7/6/2008: I previously had

from myblogapp.models
     import Post

on the second line. This works, but is inconsistent with my urls.py below and can (and did for me) cause subtle errors in the future. I corrected the line to read: from myblogsite.myblogapp.models import Post.

Modify my frontpage.html template

This is pretty self-explanatory. I access the the Post object's attributes using the "." (dot).

~/src/django/myblogsite/templates/frontpage.html:

{% extends "base.html" %}

{% block main %}
  {% for post in post_list %}
    <h2>{{ post.title }}</h2>
    {{ post.body }}
    <hr>
    <div class="post_footer">Created: {{ post.date_created }} | Tags: {{ post.tags }}</div>
  {% endfor %}
{% endblock %}

A little CSS

In between the <style> tags in ~/src/django/myblogsite/templates/base.html:

.post_footer {
  font-family: Verdana, Arial, sans-serif;
  font-size:70%;
}
hr {
  border: 0;
  color: gray;
  background-color: gray;
  height: 1px;
}

Start development server and add a couple posts

I started the development server and added a couple of posts in the Admin site:

$ cd ~/src/django/myblogsite
$ python manage.py runserver

I visited http://127.0.0.1:8000/admin, logged in, and added a couple posts.

Upload to Webfaction server

pushwebf is my alias for hg push --remotecmd /home/sofeng/bin/hg ssh://sofeng@sofeng.webfactional.com/webapps/django/myblogsite

$ pushwebf
sofeng@sofeng.webfactional.com's password:
pushing to ssh://sofeng@sofeng.webfactional.com/webapps/django/myblogsite
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 7 changesets with 20 changes to 10 files

Deploy
Logged into Webfaction:

$ cd ~/webapps/django/myblogsite
$ hg update -C
9 files updated, 0 files merged, 4 files removed, 0 files unresolved
$ ~/webapps/django/apache2/bin/restart

Here is a snapshot screenshot of version 0.0.3

The live site can be viewed at http://saltycrane.com/blog/

Django Blog project #3: Using CSS and Template Inheritance

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/06/django-blog-project-3-using-css-and/

You should be redirected in 2 seconds.

Version 0.0.1 of my new blog had no style. For 0.0.2, I added a little CSS to my Django template to make it look a little better.

Create new Django templates

I created two new Django templates, base.html and frontpage.html. base.html contains all the HTML and CSS that is common to all the web pages that will be on this site. frontpage.html is the template for the front page of my new blog. It uses the object-oriented technique of extending my base.html template. It uses the boilerplate code in base.html and then fills in sections with content specific to the front page. It is similar to server side includes, but more powerful because specific content can be inserted anywhere instead of just at the beginning or end. Django does have an include mechanism which acts like server side includes, but the extension method (also called Template Inheritance) is the preferred way of doing things.

Here is the code for frontpage.html:

{% extends "base.html" %}

{% block main %}
  {% for post in post_list %}
    {{ post }}
    <hr />
  {% endfor %}

{% endblock %}

Here is the code for base.html:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>
body {
 margin: 0;
 padding: 0;
 font-family: Georgia, Times, "Times New Roman", serif;
 color: black;
 width: 600px;
 border-right: 1px solid black;
}
#header {
 background-color: #666;
 border-bottom: 1px solid #333;
}
#sidebarLeft {
 float: left;
 width: 160px;
 margin-left: 10px;
 padding-top: 1em;
}
#mainRight {
 padding-top: 1em;
 margin: 0 2em 0 200px;
}
#footer {
 clear: both;
 background-color: #ccc;
 padding-bottom: 1em;
 border-top: 1px solid #333;
 padding-left: 200px;
}
h2.sidebar {
  font-size: 120%;
}
</style>
<title>{% block title %}Sofeng's Blog Version 0.0.2{% endblock %}</title>
</head>
<body>

<div id="header">
  <h1>Sofeng's Blog Version 0.0.2</h1>
</div>
<div id="sidebarLeft">
  {% block sidebar %}
  <h2 class="sidebar">ABOUT</h2>
  <p>This is my new blog created using <a href="http://www.djangoproject.com">Django</a>,
    a <a href="http://www.python.org">Python</a> web framework. This site is under 
    construction. My current blog is located at: 
    <a href="http://iwiwdsmi.blogspot.com">http://iwiwdsmi.blogspot.com</a>.
  </p>
  {% endblock %}
</div>
<div id="mainRight">
  {% block main %}{% endblock %}
</div>
<div id="footer">
  A <a href="http://www.djangoproject.com/">Django</a> site.
</div> 

</body>
</html>

Modify view to use new template

To use the new template, I created a new view called frontpage in ~/src/django/myblogsite/myblogapp/views.py:

from django.shortcuts import render_to_response
from myblogsite.myblogapp.models import Post

def frontpage(request):
    posts = Post.objects.all()
    post_body_list = [post.body for post in posts]
    return render_to_response('frontpage.html', 
                              {'post_list': post_body_list})

Correction 7/6/2008: I previously had

from myblogapp.models
     import Post

To complete the change, I mapped the url /blog to the new frontpage view in ~/src/django/myblogsite/urls.py:

from django.conf.urls.defaults import *
from myblogsite.myblogapp.views import *

urlpatterns = patterns(
    '',
    (r'^admin/', include('django.contrib.admin.urls')),
    (r'^myview1/$', myview1),
    (r'^blog/$', frontpage),
)

Upload project to Webfaction server

I updated my previous post to include a section on installing Mercurial. Here are my steps for pushing my changes to the server. It's a little more complicated than it should be because I was mucking with the repository, but future pushes should be much easier.

$ hg pull -u --remotecmd /home/sofeng/bin/hg ssh://sofeng@sofeng.webfactional.com/webapps/django/myblogsite
sofeng@sofeng.webfactional.com's password:
pulling from ssh://sofeng@sofeng.webfactional.com/webapps/django/myblogsite
searching for changes
adding changesets
adding manifests
adding file changes
added 5 changesets with 4 changes to 5 files (+1 heads)
not updating, since new heads added
(run 'hg heads' to see heads, 'hg merge' to merge)
$ hg merge
 local changed templates/base.html which remote deleted
use (c)hanged version or (d)elete? c
3 files updated, 0 files merged, 3 files removed, 0 files unresolved
(branch merge, don't forget to commit)
$ hg commit -m 'commit after merge'
$ hg push --remotecmd /home/sofeng/bin/hg ssh://sofeng@sofeng.webfactional.com/webapps/django/myblogsite
sofeng@sofeng.webfactional.com's password:
pushing to ssh://sofeng@sofeng.webfactional.com/webapps/django/myblogsite
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 3 changesets with 5 changes to 6 files

Deploy

With my cleanup of the mercurial repository, I didn't have to make any file changes. I just updated the working directory of my project using hg update -C and restarted the Apache web server.

$ cd ~/webapps/django/myblogsite
$ hg update -C
5 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ ~/webapps/django/apache2/bin/restart

Now, pointing my browser at http://saltycrane.com/blog/, shows new version 0.0.2 of my blog!

Here is a snapshot screenshot of my blog version 0.0.2:

Django Blog Project #2: Django Webfaction deployment

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/06/first-django-webfaction-deployment/

You should be redirected in 2 seconds.

I mentioned in my previous Django post that I'd write about hosting my new Django blog site on Webfaction. As promised, here it is.

I had some problems deploying my first site at Webfaction. First I got an error about pysqlite. I tried using Postgresql, and then I got an error about something else when trying to do a python2.5 manage.py syncdb. I think that they didn't set me up with a current Django installation. Luckily the support was prompt and helpful. I ended up reinstalling the Django application from the Webfaction Control Panel. Here are the steps I took to deploy my less than fledgling site.

Sign up for a shared hosting plan at Webfaction

I signed up for Webfaction's lowest plan at $9.50/month with no commitment. Here is their pricing plans.

Play with new account

I got a welcome email within 24 hours with the details about my account. sofeng.webfactional.com was a domain name which they provided. Visiting this address in my browser showed my the It worked! Django new project page. I also ssh'd into my shell account by using ssh sofeng.webfactional.com. The OS is CentOS5 which, I think, is based on Red Hat. It was a little different than the Ubuntu that I'm used to. I copied some of my configuration files over to make me feel more at home. On my home machine:

$ scp .bashrc sofeng@sofeng.webfactional.com:
$ scp .screenrc sofeng@sofeng.webfactional.com:
$ scp .emacs sofeng@sofeng.webfactional.com:

Reinstall Django

Note, if your Webfaction account is set up properly with Django, you shouldn't need to take these steps. Like I mentioned earlier, I had some problems with my Django installation. So I ended up reinstalling Django. It's not as hard as it sounds-- just a few clicks in the Control Panel.

In my browser, I went to the Webfaction Control Panel at https://panel.webfaction.com/.
Under "> Domains / websites", I selected "Applications".
I removed my current Django installation by clicking the icon with the minus sign.
I clicked the icon with the plus sign to add a new application.
I filled in the following information:
"Name:": django
"App type:": Django 0.96.2/mod_python 3.3.1/Python 2.5
Then I clicked the "Create" button.
Finally, I needed to specify which URL path I want to use with my Django application.
Under "> Domains / websites", I selected "Websites".
I selected the icon with the pencil to edit the website settings.
In the "Site apps:" section, I selected the icon with the plus sign to add a new App and path. I selected "django" as my "App:" and entered "/" (without the quotes) in the "URL path" field. Then I clicked "Update".
After a couple minutes, I was able to view the Django It worked! page at http://sofeng.webfactional.com.

Upload my project

The welcome message recommends using sftp, but I used rsync instead because I'm more familiar with it:
```
sofeng@home:~ $ rsync -avz ~/src/django/myblogsite sofeng@sofeng.webfactional.com:webapps/django
```

Deploy

I ssh'd into my webfaction shell account:

sofeng@home:~ $ ssh sofeng@sofeng.webfactional.com

I set the PYTHONPATH. (I actually put this in my .bash_profile)
```
sofeng@web36:~ $ export PYTHONPATH=$HOME/webapps/django
```

sofeng@web36:~ $ cd ~/webapps/django/myblogsite

I tried to do a python2.5 manage.py syncdb, but got a database error:

Traceback (most recent call last):
  File "manage.py", line 11, in 
    execute_manager(settings)
  File "/home/sofeng/webapps/django/lib/python2.5/django/core/management.py", line 1672, in execute_manager
    execute_from_command_line(action_mapping, argv)
  File "/home/sofeng/webapps/django/lib/python2.5/django/core/management.py", line 1571, in execute_from_command_line
    action_mapping[action](int(options.verbosity), options.interactive)
  File "/home/sofeng/webapps/django/lib/python2.5/django/core/management.py", line 504, in syncdb
    cursor = connection.cursor()
  File "/home/sofeng/webapps/django/lib/python2.5/django/db/backends/sqlite3/base.py", line 58, in cursor
    self.connection = Database.connect(**kwargs)
sqlite3.OperationalError: unable to open database file

I needed to edit my settings.py file. I changed the following lines:

DATABASE_NAME = '/home/sofeng/webapps/django/myblogsite/mydatabase.sqlite3'
TEMPLATE_DIRS = (
    '/home/sofeng/webapps/django/myblogsite/templates',
)

I tried python2.5 manage.py syncdb again:

$ python2.5 manage.py syncdb
Loading 'initial_data' fixtures...
No fixtures found.

It worked.

The next step was to set up the Apache configuration.
```
$ cd ~/webapps/django/apache2/conf
```

I edited the following line in httpd.conf:

    SetEnv DJANGO_SETTINGS_MODULE myblogsite.settings

Then I restarted the Apache server:
```
$ ~/webapps/django/apache2/bin/restart
```

In my browser, I went to http://sofeng.webfactional.com/myview1, but got an error:

ImportError at /myview1/
No module named blog.models
Request Method:  GET
Request URL:  http://saltycrane.com/myview1/
Exception Type:  ImportError
Exception Value:  No module named blog.models
Exception Location:  /home/sofeng/webapps/django/lib/python2.5/django/core/urlresolvers.py in _get_urlconf_module, line 177

I guess I need my project on the Python path. I edited the following line in httpd.conf:

    PythonPath "['/home/sofeng/webapps/django/myblogsite', '/home/sofeng/webapps/django', '/home/sofeng/webapps/django/lib/python2.5'] + sys.path"

Then I went back to http://sofeng.webfactional.com/myview1 and saw my page. Success!
Note, If you want to see the live site, the real location is: http://saltycrane.com/myview1. It's not very exciting now, but hopefully I will be able to add some CSS and more blog metadata soon.

Install Mercurial

Update 6/7/2008: By default Webfaction doesn't come with Mercurial installed, but they do allow you to do a user installation. I decided it would be useful to have Mercurial so I could pull and merge changes with greater control. Here are my steps.This will do a user-wide (as opposed to system-wide) installation of Mercurial in ~/bin and ~/lib.

sofeng@home:~/incoming $ wget http://www.selenic.com/mercurial/release/mercurial-1.0.1.tar.gz

sofeng@home:~/incoming $ scp mercurial-1.0.1.tar.gz sofeng@sofeng.webfactional.com:

sofeng@web36:~ $ mv mercurial-1.0.1.tar.gz ~/tmp

```
$ cd tmp
```
```
$ tar zxvf mercurial-1.0.1.tar.gz
```
```
$ cd mercurial-1.0.1
```
```
$ make install-home
```
```
$ export PYTHONPATH=$HOME/lib/python:$PYTHONPATH
```
Note: you will need to put this in your .bashrc instead of your .bash_profile because Mercurial only executes .bashrc on a remote hg push by default.

$ hg version
Mercurial Distributed SCM (version 1.0.1)

Copyright (C) 2005-2008 Matt Mackall  and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Other Errors

No module name django.core.management

$ python manage.py syncdb
Traceback (most recent call last):
  File "manage.py", line 2, in ?
    from django.core.management import execute_manager
ImportError: No module named django.core.management

You need to set the PYTHONPATH:

$ export PYTHONPATH=$HOME/webapps/django

You may get this if you use python manage.py syncdb instead of python2.5 manage.py syncdb.

Django Blog Project #1: Creating a basic blog

This is my OLD blog. I've copied this post over to my NEW blog at:

http://www.saltycrane.com/blog/2008/05/django-new-blog-project/

You should be redirected in 2 seconds.

It's been a while since my last post on Django. I became very busy but also found the Django tutorial to be somewhat dry. Luckily, the official Django Book was published during this time and it is more interesting to read. This post will be a very brief summary of the first 6 chapters of the Django Book as I apply it towards the creation of a new blog website. I highly recommend reading the book (I forgot to mention it is available free online). Then after reading the first six chapters, I hope this post can serve as kind of a refresher on how to put everything together.

As I mentioned, I decided to create my own blog site as my first Django project. I know it is not the most original idea in the world, but I thought it would be useful and a good learning experience. The following steps are my first cut at my new blog (dubbed 0.0.1) and basically just document the basic concepts of Django without providing much actual functionality.

I develop a model, a template, and a view in accordance with Django's MTV (see also MVC) development pattern. The model is of a blog post (aptly named Post) and contains only one attribute, the post body data (i.e. the actual text of the post). I should add in other data such as the title, date, tags, etc. But in order to keep things simple this first time around, it just has the post body. The model is connected to a SQLite database and is updated using Django's excellent admin interface. Finally, the model data is combined with a very basic template which just displays a title (My New Blog Version 0.0.1), and all the blog post bodies separated by a <hr>. Like I said, it's not very useful at this point, but I think I understand the basic concepts and how to put everything together much better now. The next step in my Django development will be to create some more interesting templates and views and add more useful data like titles and dates.

I also have a couple of related plans:

Set up hosting: I've decided to use WebFaction for my hosting but I need to set up and upload my new, almost-website there. This will probably be the subject of my next Django post.
Copy my Blogger posts over to my new site. I've already figured out how to use Beautiful Soup to screen scrape my Blogger posts and import them into SQLite. Likely I will do this further on in the process.

Here are the steps I took for my first cut at my new blog. Note, I'm running on Ubuntu so that's why I have the $ bash prompt and use /home/sofeng paths in my examples.

Create a new project

The first thing to do after installing Django is to create a new project. Luckily, it takes just one command to create the project.

Create a new project

$ cd ~/src/django
$ django-admin.py startproject myblogsite

Take a look at the site using the development server
```
$ python manage.py runserver
```
Then go to http://127.0.0.1:8000

Set up the Django admin interface

At first I thought the admin interface was kind of boring. However, for my blog site, I will use the admin interface to enter new blog posts.

Edit myblogsite/settings.py to add the admin application to the list of installed apps:

INSTALLED_APPS = (
   'django.contrib.auth',
   'django.contrib.contenttypes',
   'django.contrib.sessions',
   'django.contrib.sites',
   'django.contrib.admin',
)

Install database tables for the admin interface:

$ python manage.py syncdb

At this point I was asked to create a superuser to log into the admin interface. I answered "yes" and filled in the appropriate information.

Creating table auth_message
Creating table auth_group
Creating table auth_user
Creating table auth_permission
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table django_admin_log

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (Leave blank to use 'sofeng'): sofeng
E-mail address: sofeng@sofeng.com
Password:
Password (again):
Superuser created successfully.
Installing index for auth.Message model
Installing index for auth.Permission model
Installing index for admin.LogEntry model
Loading 'initial_data' fixtures...
No fixtures found.

Edit myblogsite/urls.py to include the admin url.

from django.conf.urls.defaults import *

urlpatterns = patterns('',
   (r'^admin/', include('django.contrib.admin.urls')),
)

Run the development server:
```
$ python manage.py runserver
```
Then go to http://127.0.0.1:8000/admin Log in and take a look around.

Set up the SQLite3 database

I chose SQLite because it is a lightweight, simple alternative to MySQL or PostgreSQL. This makes it great for a development website.

Edit the following section in the myblogsite/settings.py file:
```
DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = '/home/sofeng/src/django/myblogsite/mydatabase.sqlite3'
```
The rest of the DATABASE_ variables are not used with SQLite.
Test out the database configuration: Run the shell:
```
$ python manage.py shell
```
Then type these commands:
```
>>> from django.db import connection
>>> cursor = connection.cursor()
```
If nothing happens, all is good. See Table 5-2 in Chapter 5 of the Django Book common database configuration error messages.

Create an App

I think of an "app" as a piece of specific functionality of a website, whereas a project corresponds to a particular website. There can be many apps in a project. Also, apps can be used in more than one project. For more information about the differences between projects and apps see Chapter 5 of the Django Book.

Create an app

$ cd ~/src/django/myblogsite
$ python manage.py startapp myblogapp

Create a Model

I created one model, the Post model. A model roughly corresponds to a SQL table. And each attribute in that model corresponds to a table row. I added the class Admin: so that my Post model would show up in the Admin interface (where I can insert the data).

Edit myblogsite/myblogapp/models.py to look like the following:

from django.db import models

class Post(models.Model):
   body = models.TextField()

   # in the future I will add these other attributes
#    title = models.CharField(maxlength=500)
#    timestamp = models.CharField(maxlength=50)
#    tags = models.CharField(maxlength=200)

   class Admin:
       pass

Correction 7/6/2008: For the Post's body field, I previously used the line: body = models.CharField(maxlength=999999). However, thanks to Myles's comment in my post #4, I've changed this to use the more appropriate TextField.

Install the Model

After writing the Python model code, I needed to create the actual tables in the SQLite database. The following steps include a couple of checks, then I create the tables in the last step.

Edit myblogsite/settings.py file again and add the blog app to the list of installed apps:

INSTALLED_APPS = (
   'django.contrib.auth',
   'django.contrib.contenttypes',
   'django.contrib.sessions',
   'django.contrib.sites',
   'myblogsite.myblogapp',
)

Try validating the model:
```
$ python manage.py validate
```
Which gives the following message:
```
0 errors found.
```
Check the CREATE TABLE statements that Django will generate. Note, the database won't be modified.
```
$ python manage.py sqlall myblogapp
```
Which yields the following:
```
BEGIN;
CREATE TABLE "myblogapp_post" (
   "id" integer NOT NULL PRIMARY KEY,
   "body" text NOT NULL
);
COMMIT;
```
Correction 7/6/2008: I've updated the results here to reflect the correction I made to the model above.

Now, actually create the tables in SQLite:

$ python manage.py syncdb

Which yields something like this:

Creating table blog_post
Loading 'initial_data' fixtures...
No fixtures found.

Create some new data using the admin interface

Now that I created the models and tied them to the admin interface, I can start adding data using the admin interface.

Start the development server again:
```
$ python manage.py runserver
```
Go to http://127.0.0.1:8000/admin and log in.
Under the "Blog" heading, click "Posts", then add some new posts using "Add post" and the "Save" links. This will add data to the SQLite database.

Create a template

Now I will display the data I just created using a template and a view. The template holds all the HTML code and some simple Django template code which the view's Python code uses to customize the page.

Create the file myblogsite/templates/mytemplate.html and put the following inside:

<html>
 <head><title>Post</title></head>
 <body>
   <h1>My New Blog Version 0.0.1</h1>

   {% for post in post_list %}
   {{ post }}
   <hr />
   {% endfor %}

 </body>
</html>

Edit myblogsite/settings.py again to instruct Django where to find the template files.
```
TEMPLATE_DIRS = (
   '/home/sofeng/src/django/myblogsite/templates',
)
```
Be sure to include the comma at the end.

Create a view

The view is where I will grab the data from my model and insert it into my template.

Create a new file myblogsite/myblogapp/views.py and put the following inside:
```
from django.shortcuts import render_to_response
from myblogsite.myblogapp.models import Post

def myview(request):
   posts = Post.objects.all()
   post_body_list = [post.body for post in posts]
   return render_to_response('mytemplate.html',
                             {'post_list': post_body_list})
```
Correction 7/6/2008: I previously had from myblogapp.models import Post on the second line. This works, but is inconsistent with my urls.py below and can (and did for me) cause subtle errors in the future. I corrected the line to read: from myblogsite.myblogapp.models import Post.

Map an URL to the new view

Finally, I map an URL to my newly created view.

Edit myblogsite/urls.py so that it looks like:

from django.conf.urls.defaults import *
from myblogsite.myblogapp.views import myview

urlpatterns = patterns('',
   (r'^admin/', include('django.contrib.admin.urls')),
   (r'^myview/$', myview),
)

Take a look at the new page: Run the server:
```
$ python manage.py runserver
```
Then go to http://127.0.0.1:8000/myview Visiting the url shows all the posts I entered through the admin interface. Nice. Here is a snapshot screenshot of my new blog:

That's it for now. I tried to map out the basic steps for using Django's MTV development pattern. Hopefully, in the future, I'll be able to add more useful features to my new Django-powered blog.

Sofeng's Old Blog

Django Blog Project #9: Migrating Blogger posts with Beautiful Soup

Django Blog Project#8: Adding basic comment functionality with Django Free Comments

Django Blog Project #7: Adding a simple Atom feed

Django Blog Project #6: Creating standard blog views

Django Blog Project #5: YUI CSS and serving static media

Django Blog Project #4: Adding post metadata

Django Blog project #3: Using CSS and Template Inheritance

Django Blog Project #2: Django Webfaction deployment

Django Blog Project #1: Creating a basic blog

About