thefekete.net

$> :(){ :|:& };:

Remove trailing whitespace from lines in Vim

Put this in your command mode and smoke it:

:s/ \+$//g

Match RAR archives

The Following can be used to find the first RAR file in a multi-volume archive. It will find *.part01.rar or *.rar depending on the naming convention in use.

.*[^p][^a][^r][^t]..\.rar$|.*part01\.rar$

If you need to find all the RAR files and SFV files (probably to delete them after extraction), the following will get them all.

.*\.r[0-9]{2}$|.*\.rar$|.*\.sfv$

Efficiently testing multiple model methods at once in Django

Update 2011-08-03

I have since ditched this idea because “Explicit is better than implicit” and “Readability counts.”

The method listed below is not very readable and very implicit in nature. Basically, if you handed this to someone (or you in 6 months), it wouldn’t be immediately obvious what is doing what. Which is not the reaction you want when tests are failing for some unknown reason.

I’ve settled on separate test methods for each model method/attribute. It’s a little more verbose, but very clear and easy to decipher.

I have left the original post to document my learning process, and to remind myself not to cross that fine line between clever and stupid.

/Update

I recently wrote some tests for a model that had many (eight) methods that calculated various values from the fields of the model. I needed to test the methods under many different conditions and found myself writing eight or so self.assertEqual(...) statements for each test condition. I was repeating myself and I didn’t like it.

To remedy the situation, I made a list of the methods I was calling over and over, and then wrote a simple function to iterate over the listed methods and assert that they returned equal to the values I passed the function. Below is a simplified example of how it worked:

# tests.py

from models import MyModel

class MyModelTest(TestCase):

    def test_methods(self):
        methods = ('method1', 'method2', 'method3')

        def assert_methods(obj, values):
            for m, v in zip(methods, values):
                ret = getattr(obj, m)()
                try:
                    self.assertEqual(ret, v)
                except AssertionError:
                    raise AssertionError('%s[%s].%s() returned %s expected %s'
                            % (obj.__class__.__name__, obj.pk, m, ret, v))

        # Create an object to test
        obj1 = MyModel.objects.create(...)
        # Assert that method1(), method2() and method3 return
        # 123, 'abc', and None respectively
        assert_methods(obj1, (123, 'abc', None))
        # Make some change to obj1
        obj1.somefield = 18
        obj1.save()
        # Assert the change on the methods
        assert_methods(obj1, (987, 'abc', False))

        # Create a different object
        obj2 = MyModel.objects.create(...)
        # Assert different return values for methods
        assert_methods(obj2, (456, 'efg', False))

The vital parts above are the methods list and assert_methods(). methods stores a list of strings naming the methods you want to call on your model instance. assert_methods() takes an object (the model instance) and a list of values to compare to the method’s returns.

The error handling prints out a more helpful message when assertion fails, including the model name, primary key, and method called along with the expected and actual values.

So basically, assert_methods() asserts that methods[0] = values[0], methods[1] = values[1] and methods[n] = values[n], etc.

Of course, this is limited to one type of assertion for all the methods, but it’s really useful when you gotta do the same thing a bunch of times. Anyways, hope this helps someone out there, if so let me know.

Quickly check line character maximums from the command line

If you want to make sure you didn’t go over 79 or 80 characters in a file you can simply run:

#!/bin/bash
grep -Pn '.{80,}' file-to-check

This uses grep’s perl regex option (-P) so we can use the expression .{80,} to find any lines that have >= 80 characters. The -n option prints the line numbers in the output so you can find them easily.

Initialize a remote git repository from an existing local repo

#!/bin/bash
REMOTE_HOST='example.com'
REMOTE_PORT=22
REPO_NAME='my_repository'

# create the bare remote repo
ssh $REMOTE_HOST -p$REMOTE_PORT \
"mkdir -p /var/git/$REPO_NAME.git && \
cd /var/git/$REPO_NAME.git && git init --bare" &&

# configure the local repo
git remote add origin ssh://$REMOTE_HOST:$REMOTE_PORT/var/git/$REPO_NAME.git &&
git config branch.master.remote origin &&
git config branch.master.merge refs/heads/master &&

# see if it worked...
git push origin master

Project Pointers

Got this from a comment on slashdot by raddan. Seems like good advice…

  • Never pass unvalidated input to your database
  • Never pass unvalidated input to the system
  • Always validate on the server-side; client-side validation should only function as a convenience to the user
  • Validate data coming from other servers (if you’re doing any web services stuff).
  • Encrypt connections to the server
  • Enforce inactivity timeouts
  • Do not allow multiple logins to the same account (unless you want your game to application to work that way)
  • Always authenticate users; consider using two-factor authentication (CAPTCHA + password, etc)
  • Allow administrators to revoke accounts
  • Make it easy for administrators/force administrators to sandbox/chroot your application
  • If your applications needs to use server storage, consider DoS attacks (a user uploading lots of stuff)
  • Make sure all privileged actions hit the same authentication class/function; if you change your authentication code, this ensures that the changes are applied across the board <– I catch newbie programmers making this mistake all the time!

If you do all of the above, your app might still not be “secure”, but breaking it will be a PITA.

Django: Extending Model Queryset / Manager Methods

Django models automatically come with some very useful methods under the objects Manager. Methods like Model.objects.all(), Model.objects.filter() and Model.objects.get() work quite well.

But what if you are constantly using the same QuerySet over and over? You can either keep typing MyModel.objects.filter(way-to-many-kwargs).order_by(something).etc... over and over -or- extend the objects Manager with your own crazy method.

Let’s say you’ve got the following Model:

class MyModel(models.Model):
    ...
    ...
    ...

All you need to do is subclass Manager, add your method and override your model’s old Manager:

class MyModelManager(models.Manager):
    def crazy_query(self):
        ...
        # A bunch of queries, joins, logic, etc...
        ...

class MyModel(models.Model):
    ...
    ...
    ...

    objects = MyModelManager()

Enabling AdminDocs in Django

In order to make a handy Documentation link available in the upper right portion of the Django admin site, simply add the admindocs app to your settings.py and the admindocs urls to your urls.py

# In your settings.py:
INSTALLED_APPS = (
    ...
    'django.contrib.admindocs',
    ...
)

# In your urls.py:
urlpatterns = patterns('',
    ...
    (r'^admin/doc/', include('django.contrib.admindocs.urls')),
    (r'^admin/', include(admin.site.urls)),
    ...
)

NOTE: The admindocs urlconf must precede the admin site urlconf to work properly.

Sorting Hierarchical Categories in Django

I had almost given up on my dream of hierarchical (multi-level) categories sorted by their self-referential parent to form a tree. Almost, but I finally figured it out.

I’m not sure if it’s a hack or the “one obvious way”, as Tim Peters puts it (probably not), but it works and I’m using it.

The whole problem starts with a simple model:

class Category(models.Model):
    name = models.CharField(max_length=100)
    parent = models.ForeignKey('self', blank=True,
        null=True, related_name='child')

    def __unicode__(self):
        if self.parent:
            prefix = str(self.parent)
        else:
            return self.name
        return ' > '.join((prefix,self.name))

It works great for categories with sub categories. And when you call the __unicode__() method, it even spits out the whole ancestry to the category (eg. u'GrandParent > Parent > Category').

But alas, as soon as you try to sort them by their parents, you find out that the full name is not a real field and you can’t order by fake fields. Of course you might get the bright idea to add the order_with_respect_to = 'parent' Meta class attribute, but that will only lead you here.

Read the rest of this entry »

Slug urlconf regex pattern for Django urlpatterns

If you want to match a slug in your urls.py, use (?P[-\w]+) as a named match pattern:

So, if you wanted to match http://example.com/some-slug-name/, you would use

urlpatterns = patterns('',
    # ...
    url(r'^(?P<slug>[-\w]+)/$', some_view),
    # ...
)

I got this info from a Django Users Thread on google groups.