Nov 30, 2011

Вывод прошедшего времени в удобочитаемом виде на JavaScript

function date_from_string(dt){
 // 2011-11-30 15:40:50
 var df = dt.split(' ');
 var d = df[0].split('-');
 var t = df[1].split(':');
 return d1 = new Date(d[0],d[1]-1,d[2],t[0],t[1],t[2]);
}

function diff_date(d1){
 var r = (new Date() - d1)/1000;
 
 var tt = {
   sec: ['{} секунд','{} секунда','{} секунды'],
   min: ['{} минут','{} минута','{} минуты'],
   hour: ['{} часов','{} час','{} часа'],
   day: ['{} дней','{} день','{} дня']
 }
 function sec(x,dtt){
  var r;
  x = x.toFixed(0);
  if(x>=11 && x<=14) r = null
  else {
   var s = '' + x;
   if(s.length>1) s = s.substring(1);
   r = { '1':dtt[1], '2':dtt[2], '3':dtt[2], '4':dtt[2] }[s];
  }
  if(!r) r = dtt[0];
  return r.replace('{}',x)
 }
 
 if(r<60) return sec(r, tt.sec) + ' назад';
 r = r / 60;
 if(r<60) return sec(r, tt.min) + ' назад';
 r = r / 60;
 if(r<24) return sec(r, tt.hour) + ' назад';
 r = r / 24;
 if(r<1) return 'сегодня';
 if(r<2) return 'вчера';
 return sec(r, tt.day) + ' назад';
}

Nov 11, 2011

Python Enum

class Color(Enum):
    RED = EnumValue('R', 'Red')
    GREEN = EnumValue('G', 'Green')
    BLUE = EnumValue('B', 'Blue')
Where Enum from here django-stdfields models.py

Nov 10, 2011

Django applications

CMS

Mezzanine - An open source content management platform built using the Django framework.
django-fiber - a simple, user-friendly CMS
merengue - Merengue is a full-featured and pluggable CMS designed for creating websites, writing less code and mantaining clean, elegant and re-usable code.
feincms - A Django-based CMS with a focus on extensibility and concise code

Blog

django-multiblogs - An application for managing multiple blogs, and even *gasp* blog "sets".
django-articles - The blog engine that powers codekoala.com. See http://bitbucket.org/codekoala/django-articles/issues for the ticket tracker

Forum

pybbm - PyBB Modified. Django forum application

Feedback

django-basic-feedback - Provides a Feedback button on your pages and lets you view feedback via Django's admin site. Requires jQuery. 
Django-feedback

Shop

Plata - the lean and mean Django-based Shop
LFS - Lightning Fast Shop

Shopping cart

Cartridge -A Django shopping cart application

Categories

django-categories - This app attempts to provide a generic category system that multiple apps could use. It uses MPTT for the tree storage and provides a custom admin for better visualization

Wiki

django-wiki - Super simple pluggable wiki application for Django.
Hatta - Wiki engine that lives in Mercurial repository.

Poll

django-poll-system - The application to organize polling (or voting) on your site. Demo

Admin

django-admin-sortable - Drag and drop sorting for models and inline models in Django admin
django-admin-filtrate - This Django app makes it easier to create custom filters in the change list of Django Admin and supplies a TreeFilter and a DateRangeFilter too
django-exportable-admin - Provides a custom ModelAdmin which adds CSV export for changelist views

Hosts

django-subdomains - Subdomain tools for the Django framework, including subdomain-specific URL routing
django-hosts - Dynamic and static host resolving for Django. Maps host names to URLconfs

Db

django-denorm - provides a declarative way of denormalizing models in Django based applications while maintaining data consistency.

Permissions

django-object-permissions - A method for adding object-level or row-level permissions
django-moderation - is reusable application for Django framework, that allows to moderate any model objects.
django-su - Login as any user from the Django admin interface, then switch back when done

Messages

django-postman - User-to-User messaging system for Django, with gateway to AnonymousUser, moderation and thread management, user & exchange filters, auto-complete support.

Translation

django-transmeta - an application for translatable content in Django's models. Each language is stored and managed automatically in a different column at database level.

Template tag

django-ttag - A template tag constructor library for Django

Image

django-image-cropping - A reusable app for cropping images easily and non-destructively in Django

JavaScript and CSS Compression

django-pipeline - Pipeline is an asset packaging library for Django, providing both CSS and JavaScript concatenation and compression, built-in JavaScript template support, and optional data-URI image embedding.

Geo

django-cities - Countries and cities of the world for Django projects
django-countries - Nice little application for Django projects providing fixtures and models for a "complete" list of world countries

Map

django-leaflet - allows you to use Leaflet (version 0.3) in your Django projects.

PDF

pdfdocument - ReportLab-wrapper

Barcodes

python-qrcode - This module uses the Python Imaging Library (PIL) to allow for the generation of QR Codes.

CSV

django-csv-importer - Convert csv files into python object or django model
django-exportable-admin - Provides a custom ModelAdmin which adds CSV export for changelist views

Visitors analytic

httpagentparser - Extracts OS Browser etc information from http user agent string
django-mobi - Django middleware and view decorator to detect phones and small-screen devices.
geobaza - Geobaza is the software for geolocation by IP-address.
django-online-counter - Django online visitor counter

JavaScript

djangbone - Django backends to talk to Backbone.js
dajaxproject - Easy to use AJAX libraries for django

Autocomplete/Select

django-ajax-selects - jQuery UI-powered auto-complete fields for ForeignKey, ManyToMany and text fields
django-chosen - django FormFields using the Chosen javascript plugin for jQuery
django-smart-selects - chained and grouped selects for django forms

Inline edition

django-inplaceedit - allows inline edition of some data from the database

File upload

plupload - The developers of TinyMCE brings you Plupload, a highly usable upload handler for your Content Management Systems or similar. Plupload is currently separated into a Core API and a jQuery upload queue widget this enables you to either use it out of the box or write your own custom implementation.
uploadify - Uploadify is a jQuery plugin that integrates a fully-customizable multiple file upload utility on your website. It uses a mixture of Javascript, ActionScript, and any server-side language to dynamically create an instance over any DOM element on a page.

JSON

django-serverpush - django server push solution

GUI

pyside - bindings for the Qt cross-platform application and UI framework
wxpython - a blending of the wxWidgets C++ class library with the Python programming language

Testing

selenium - Python bindings for Selenium
django-selenium - Selenium testing support for django

Logging

django-peavy
  • Middleware to tag each request with a unique ID.
  • Logging filters to capture request metadata like user, remote IP, and headers.
  • Logging handlers for:
    • Capturing exception information, including a copy of the Django server error page, in a database.
    • Sending error notifications to admins without revealing sensitive information like the contents of request.POST.
  • A database router for sending log records to a separate database.
  • A simple user interface for browsing log records in the database.
django-object-log - A method for logging user actions on models

Profiling

django-profiler - Util for profiling python code mainly in django projects but can be used also on ordinary python code. It counts sql queries a measures time of code execution.
django-debug-logging - A plugin for the Django-Debug-Toolbar to provide statistic logging and a UI for reviewing the logs.
django-statsd - Integration between statsd and django. It allows you to use different clients, sends timings as middleware and integrates with django debug toolbar.
psutil - is a module providing an interface for retrieving information on all running processes and system utilization (CPU, disk, memory, network) in a portable way by using Python, implementing many functionalities offered by command line tools.

Data visualization

django-graphviz

Django and Python Coding Style and Best Practices

Introduction:
The Zen of Python (PEP 20)

Follow guide lines:
Style Guide for Python Code (PEP 8)
Django coding style
Google Python Style Guide (RU)

Tutorials:
Django best practices fork of Django Reusable App Conventions
Django Design Patterns
Django Advice
The Hitchhiker’s Guide to Python!
Python Project Howto
Code Like a Pythonista: Idiomatic Python
Nick Coghlan’s Python Notes
A Guide to Python's Magic Methods

Code checkers:
django-lint Online

Learn by examples:
Rosettacode code and algorithms examples

Be aware of
Python Standard Library: A Few of My Favorite (Python) Things
Design patterns
http://www.pypedia.com/index.php/Main_Page


Thoughts on RESTful API Design

Setup module function (UnitTest)

The default ordering of tests created by the unittest test loaders is to group all tests from the same modules and classes together. This will lead to setUpClass / setUpModule (etc) being called exactly once per class and module. If you randomize the order, so that tests from different modules and classes are adjacent to each other, then these shared fixture functions may be called multiple times in a single test run.
import unittest

def setUpModule():
    print 'Module setup...'

def tearDownModule():
    print 'Module teardown...'

class Test(unittest.TestCase):
    def setUp(self):
        print 'Class setup...'

    def tearDown(self):
        print 'Class teardown...'

    def test_one(self):
        print 'One'

    def test_two(self):
        print 'Two'

Nov 9, 2011

Mercurial HowTo

Tutorial
http://hginit.com/01.html

How to correctly close a feature branch in Mercurial?
hg up feature-x
hg ci -m 'branch closed' --close-branch

hg up default
hg merge feature-x
hg ci -m merge
To completely undo the uncommitted merge and discard all local modifications, you will need to issue a
# note the "dot" at the end of the command
hg update -C -r .

Merge or rebase with uncommitted changes

hg diff > somefile.diff
hg revert -a
hg import --no-commit somefile.diff

Undo not pushed commit

hg rollback

Finding the nearest with the ORM

from django.contrib.gis.geos import Point
lawrence = Point((-95.234657999999996, 38.972679999999997))
Campground.objects.all().distance(lawrence).order_by('distance')

Django decorator/middleware cache key for given URL

import hashlib
from django.utils.encoding import iri_to_uri
from django.conf import settings
from django.utils.translation import get_language

def url_cache_key(url, language=None, key_prefix=None):
    if key_prefix is None:
        key_prefix = settings.CACHE_MIDDLEWARE_KEY_PREFIX
    ctx = hashlib.md5()
    path = hashlib.md5(iri_to_uri(url))
    cache_key = 'views.decorators.cache.cache_page.%s.%s.%s.%s' % (
        key_prefix, 'GET', path.hexdigest(), ctx.hexdigest())
    if settings.USE_I18N:
        cache_key += '.%s' % (language or get_language())
    return cache_key


http://djangosnippets.org/snippets/2595/

Nov 8, 2011

The Art of Readable Code - D. Boswell and T. Foucher

  • Attach important details to variable names - for example, append _ms to a variable whose value is in milliseconds or prepend raw_ to an unprocessed variable that needs escaping.
  • Use longer names for larger scopes - don’t use cryptic one- or two-letter names for variables that span multiple screens; shorter names are better for variables that span only a few lines.

Word Alternatives

send - deliver, dispatch, announce, distribute, route
find - search, extract, locate, recover
start - launch, create, begin, open
make - create, set up, build, generate, compose, add, new

Avoid generic names like tmp and retval, unless there’s a specific reason to use them.

Values with Units

Start(int delay) - delay - delay_secs
CreateCache(int size) - size - size_mb
ThrottleDownload(float limit) - limit - max_kbps
Rotate(float angle) - angle - degrees_cw

Encoding Security Attributes

This technique of attaching extra information to a name isn’t limited to values with units. You should do it any time there’s something dangerous or surprising about the variable.

For example, many security exploits come from not realizing that some data your program receives is not yet in a safe state. For this, you might want to use variable names like untrustedUrl or unsafeMessageBody. After calling functions that cleanse the unsafe input, the resulting variables might be trustedUrl or safeMessageBody.
  • A password is in “plaintext” and should be encrypted before further processing - password- plaintext_password 
  • A user-provided comment that needs escaping before being displayed - comment - unescaped_comment 
  • Bytes of html have been converted to UTF-8 - html - html_utf8 
  • Incoming data has been “url encoded” - data - data_urlenc

Coments Markers

TODO: Stuff I haven’t gotten around to yet
FIXME: Known-broken code here
HACK: Admittedly inelegant solution to a problem
XXX: Danger! major problem here

Variables and readability

  • Reduce the scope of each variable to be as small as possible. Move each variable to a place where the fewest lines of code can see it. Out of sight is out of mind.
  • Prefer write-once variables. Variables that are set only once (or const, final, or otherwise immutable) make code easier to understand.

Reorganizing Your Code

  • Separate the generic code from the project-specific code.
  • Code should be organized so that it’s doing only one task at a time.
  • Avoid writing new lines of code by:
    • Eliminating nonessential features from your product and not overengineering
    • Rethinking requirements to solve the easiest version of the problem that still gets the job done
    • Staying familiar with standard libraries by periodically reading through their entire APIs

Testing and Readability

  • Implementing Custom “Minilanguages
  • Use the simplest test inputs that completely exercise your code.
  • Give your test functions a fully descriptive name so it’s clear what each is testing. Instead of Test1(), use a name like Test_<FunctionName>_<Situation>.

Data apps for Django

django-tables2

Simplifies the task of turning sets of data into HTML tables. It has native support for pagination and sorting

{% load django_tables2 %}
{% render_table table %}

django-autoreports (csv export)

Autoreports is a Django application that lets you create reports in very Django Admin listing
Redefine 'admin/change_list.html' templates as follows
{% load adminmedia admin_list i18n autoreports_tags %} {# add auto_reports_tags #}

#...#

{% block object-tools %}
  {% if has_add_permission %}
    <ul class="object-tools">
      <li>
        <a href="add/{% if is_popup %}?_popup=1{% endif %}" class="addlink">
          {% blocktrans with cl.opts.verbose_name as name %}Add {{ name }}{% endblocktrans %}
        </a>
      </li>
      {% autoreports_admin %} {# add this stuff #}
    </ul>
  {% endif %}
{% endblock %}

#...#
For basic usage you have to api, and you can export to CSV each models.
/autoreports/app_label/module_name/
/autoreports/app_label/module_name/?filter1=value1

django-easyfilters

Easy creation of link-based filtering for a list of Django model objects. Demo

django-datatables - datatables.net

Nov 7, 2011

django-sekh - Highlight the keywords of a page if a visitor is coming form a search engine.

django-fabfile

django-fabfile - Fabric tasks for Django and Amazon Web Services

Django soft delete

django-softdelete Soft delete for Django ORM, with support for undelete

Forms apps, snippets, widgets

django-forms-ext

Fields
  • ForeignKeyChoiceField - When you have a formset that has a foreign key, Django will fire off a new (identical) query to build the choices for that field for each form in the formset by default. Using this field will allow you to run the query once for the choices and re-use that queryset for each form in the formset. CommaSeparatedField - Django has a comma separated integer field, but not just strings. Stupid, I know... we created one for use with strings 
  • QuerysetChoiceField - When you want to build a select box with a queryset but don't want a model instance when saving, use this thing. 

Views
  • FormSetView - Django 1.3's generic views do not include a FormSet view. That's what this is.

django-stdfields

Fields
  • MinutesField: use an integer to represent a duration of minutes and hours
  • EnumIntegerField: makes working with choices a bit easier
  • EnumCharField: the same, but for choices with a char key

django-forms-builder

A Django reusable app providing the ability for admin users to create their own forms

django-bfm

  • Multifile Uploads (Open File Dialog)
  • Live Upload Status report (Upload applet)
  • File browsing
  • Directory support
  • Core features has no dependencies (except for Django), lightweight
  • Looks like django admin (extends admin template)

django-custom-field

Allow end users to create easy (but slower to work with) fields attached to any Django model. Includes support so fields show up in the admin interface and helper functions that make it easy to access any custom fields programmatically.

django-wizard

allows views to be organized into steps for linear page flows

django-easyfilters

http://pypi.python.org/pypi/django-easyfilters/ - Easy creation of link-based filtering for a list of Django model objects.

DataTables jQuery plugin

DataTables jQuery plugin

Nov 3, 2011

geobaza - Geobaza is the software for geolocation by IP-address.

Nov 2, 2011

Snippets, schemas

Schema.org - This site provides a collection of schemas, i.e., html tags, that webmasters can use to markup their pages in ways recognized by major search providers. Search engines including Bing, Google and Yahoo! rely on this markup to improve the display of search results, making it easier for people to find the right web pages.

Python / Django Performance profiling

https://code.djangoproject.com/wiki/ProfilingDjango
http://ianozsvald.com/2012/03/18/high-performance-python-1-from-pycon-2012-slides-video-src/
Профилирование и отладка Python, инструменты

Django

django-extensions - this is a repository for collecting global custom management extensions for the Django Framework
python manage.py runprofileserver --use-cprofile --prof-path=/tmp/output
django-perftools
  • QueryCountLoggingMiddleware - Perftools includes a logger that will monitor requests execution time. Once it hits the defined threshold, it will log to the named perftools logger, including the metadata for the request (as defined by Sentry's logging spec).
  • RemoteProfilingMiddleware - Profiles a request and saves the results to disk.
  • SlowRequestLoggingMiddleware - Logs requests which exceed a maximum number of queries.
django-profiler is util for profiling python code mainly in django projects but can be used also on ordinary python code. It counts sql queries a measures time of code execution. It logs its output via standard python logging library and uses logger profiling. If your profiler name doesn't contain any empty spaces e.g. Profiler('Profiler1') django-profiler will log all the output to the profiling.Profiler logger. @profilehook decorator uses profilehooks python package to gather code execution stats. Except it logs via standard python logging it also outputs code execution stats directly to sys.stdout.
from profiling import profile

@profile
def complex_computations():
    #some complex computations

django-processinfo - application to collect information about the running server processes.

dogslow - Dogslow is a Django watchdog middleware class that logs tracebacks of slow requests.

Python

line_profiler - is a module for doing line-by-line profiling of functions. kernprof is a convenient script for running either line_profiler or the Python standard library's cProfile or profile modules, depending on what is available.
runsnakerun - is a small GUI utility that allows you to view (Python) cProfile or Profile profiler dumps in a sortable GUI view. It allows you to explore the profiler information using a "square map" visualization or sortable tables of data. It also (experimentally) allows you to view the output of the Meliae "memory analysis" tool using the same basic visualisations.
dis - The dis module supports the analysis of CPython bytecode by disassembling it. The CPython bytecode which this module takes as an input is defined in the file Include/opcode.h and used by the compiler and the interpreter.
def myfunc(alist):
    return len(alist)

>>> dis.dis(myfunc)
  2           0 LOAD_GLOBAL              0 (len)
              3 LOAD_FAST                0 (alist)
              6 CALL_FUNCTION            1
              9 RETURN_VALUE

plop - Plop is a stack-sampling profiler for Python. Profile collection can be turned on and off in a live process with minimal performance impact.
statprof.py - This package provides a simple statistical profiler for Python.
pytrace - is a fast python tracer. it records function calls, arguments and return values. can be used for debugging and profiling.

WSGI

wsgi-shell - The 'ispyd' package provides an in process shell for introspecting a running process. It was primarily intended for investigating running WSGI application processes, specifically to determine what a process is doing when it hangs, but has many other features as well. This includes being able to start an embedded interactive Python interpreter session, set debugger probe points to record tracebacks for exceptions and then later run 'pdb' in post mortem mode on those exceptions.

Linux

top -H
1 - нагруженность цпу
С - сортировать по цпу
М - сортировать по памяти
к - прибить процесс
с - показать путь к комманде

vmstat - выдает информационный отчет о активности процессов, памяти, свопинга, поблочного ввода/вывода, прерываний и процессора
w - кто зарегистрирован и что они делает
free – использование памяти
pstree - процессы в виде иерархии

ps
ps -u someusername -o pid,%cpu,%mem,start_time,size=-size-,state,cmd
ps -u someusername -o pid,%cpu,%mem,start_time,size=-size-,state,comm | grep runfastcgi.fcgi

Disks usage
df -h

Nov 1, 2011

Images processing apps

django-imagekit
from django.db import models
from imagekit.models import ImageSpec
from imagekit.processors import resize, Adjust

class Photo(models.Model):
    original_image = models.ImageField(upload_to='photos')
    thumbnail = ImageSpec([Adjust(contrast=1.2, sharpness=1.1),
            resize.Crop(50, 50)], image_field='original_image',
            format='JPEG', quality=90)

Translation apps

django-linguo

from linguo.models import MultilingualModel
from linguo.managers import MultilingualManager

class Product(MultilingualModel):
    name = models.CharField(max_length=255, verbose_name=_('name'))
    description = models.TextField(verbose_name=_('description'))
    price = models.FloatField(verbose_name=_('price'))

    objects = MultilingualManager()

    class Meta:
        # name and description are translatable fields
        translate = ('name', 'description')