The world (and word) of jaymz

Two things I use a lot at work – Django and Active Collab. One is the python framework we build everything on and the other is a project management tool that we use – think a local install of Basecamp roughly.

It can get annoying when we create development versions of things we’re working on and have to go and create test users for people; so I thought since writing an auth backend for django is so easy why not just use that to allow any user with a valid Active Collab account to login to the dev admin.

The code is on github and will automatically create a superuser in django’s auth table. Users can then login with their active collab emails & passwords without having to pass around test accounts. It also means users wont suddenly lose access when you nuke the database for imports saving you some earache. Just add your base active collab url to the AC_URL variable in settings.py and add ‘acollabauth.backends.ActiveCollabBackend‘ to your AUTHENTICATION_BACKENDS tuple.

I have also blogged about this over at udox.

geoip-redirect is available on my github page.

Something that I’ve had need of more than once the past while is placing lots of address data on google maps. For a few places it’s easy to manually handle that but I’ve been having to work with databases with thousands and thousands of samples – many in an unclean form with fields missing.

I’ve ended up with a small django based library over at github. For the data I had to work with much of the complications where the mis-matching of city/town level names – for example as well as London many items had the actual area listed instead, e.g. Camden or Westminster. When doing queries we wanted to be able to return items within certain areas but without having to manually clean everything up, so the model code provides a within_radius method which will return matching enteries that fall within the radial bounds given.

Within the scripts folder there’s an example of how I decided to work with an existing django based site that I then installed the geosearch code to.

Code block

#!/usr/bin/python2.7
# -*- coding: utf-8 -*-
 
# To run this you'll need to set two environment variables. Could probably do this
# as a management command but seems a bit messy since it's a very specific thing...
#
# export PYTHONPATH=/home/jaymz/development/www/vansemea
# export DJANGO_SETTINGS_MODULE=settings
#
# Obviously you'll need to change that path if you want to run it yourself
 
from locator.models import Dealer, City
from geosearch.models import GeoEntry
from base.models import Country
from django.db.models import Count
 
MAX_TOP_CITIES = 30 # how many of our "has the most dealers" to bin the others into
BOUNDRY_RADIUS = 25 # number of *miles* from our source point to include
CTYPES = {
    'dealer': 32,
    'city': 29,
}
 
for country in Country.objects.all():
    try:
        cities = Dealer.objects.values('city__pk', 'city__name') \
            .filter(country=country).annotate(Count('city')) \
            .order_by('-city__count')[:MAX_TOP_CITIES]
    except IndexError:
        print "No data for %s" % country.name
 
    print "Working out cities for %s\n" % country
 
    for city in cities[::-1]:
        # Look up the GeoEntry for this city. If we have one then we'll get
        # back a list of other enteries which are within BOUNDRY_RADIUS miles
        print "\nGetting db places within %dmiles of %s...\n" % (BOUNDRY_RADIUS, city['city__name'])
 
        try:
            primary_city = City.objects.get(pk=city['city__pk'], country=country)
            primary_city.primary_city = primary_city
            primary_city.save()
        except:
            "Could not find city matching %d and country id %d" % (city['city__pk'], country.pk)
            continue
 
        try:
            city_geoentry = GeoEntry.objects.get(content_type__pk=CTYPES['city'],
                object_id= city['city__pk'])
        except GeoEntry.DoesNotExist:
            print "No geoentry for %s" % city['city__name']
            continue
 
        radial_results = GeoEntry.within_radius(
            [city_geoentry.latitude, city_geoentry.longitude],
            BOUNDRY_RADIUS
        )
 
        # Trim off our source point, reverse and update our source cities
        # (we need to reverse it to have it assign by most populated last)
        for point in radial_results[1:][::-1]:
            if point['content_type'] == CTYPES['city']:
                source_city = City.objects.get(pk=point['object_id'])
                source_city.primary_city = primary_city
                source_city.save()
                print "Set %s as primary city for %s" % (primary_city, source_city)

sigh… Recently I have had to (more than once) explain why we’re choosing Django to develop a e-commerce & web app platform rather than using something off the shelf like Magento. I’ve heard lots of negative things regarding Magento and here’s what I’ve outlined in an email to PM’s etc which others might find useful if they have a similiar “justify it” experience.

The number one reason though I’d rather not work with Magento is that time and again developers express difficulty in making magento work for them and not their magento consultant.

Anywayz, I’m sure someone will take offense to this and I’m well aware I’m comparing a framework to a specific application but whatever…

Actually come to think of it, I’m not even comparing it really django, just listing some gripes and hate I’ve read/heard elsewhere

Code extension & extendability

June 2009 : pickledshark.com

This guy is basically critical of the codebase being complex and difficult to work with. He ends with this:

Due to the Zend/OOP/MVC influence on Magento it is impossible to follow the code. Classes are referenced dynamically, various aspects are contained in XML files and there is no clear flow that you can just debug through. The sheer volume of files and folders makes finding something unbelievably tedious.

Even the database is a minefield. In every other system I have used finding data is easy. In Magento, the use of EAV means that data is split amongst hundreds of abstract tables. Again, it doesn’t flow and it doesn’t make sense without a great deal of time developing a solid understanding of what they have done.

A final Cynical Note… Many people claim that the complexity of Magento is somewhat intentional. The profitability of Magento relys on consulting, technical support and installations. Making the codebase complex could mean that many developers will start out, get stuck and pay for help. If this is the intention of Varien then perhaps they have been very goal focused

Hmm…… Another post here says that for most bits that arent in the free or enterprise version you can probably find a plugin to do it for you at magento-connect. Here’s something you’d imagine would be useful “get the lowest price free” and there is an extension for it. Problem being its $150.
Lowest Price Free plugin

The comments that I find over & over are that the system itself is a nightmare to work from from a developer viewpoint unless you are some magento expert consulting for a fee (big surprise). Eg, here: (april 2009)

The big thing for me right now is the documentation(specifically around customization) – it is nearly non-existent. The forums have TONS more questions that(sic.) answers, most of the Wiki articles focus on simple store administration or VERY simple theme level customizations (via CMS pages or edits to layouts).

Multiple products with similiar data & attributes (eg sizes)

Last post 1 HOUR AGO!

A long discussion about how much of a ball-ache it is to manage multiple products that share a common base (i.e. different sizes of the same product). There is some talk about configurable products, grouped products etc but it seems that even all these still require duplication/entry of the same data per SKU:

This problem is driving me nuts! I have a shop to build with over 100,000 possibilities of one single configurable product. Now… Could anyone tell me, how exactly I am supposed to manage this? I just can’t seemto find anyone dumb enough to add those products manually and I’m certainly not doing it myself! This is magento’s great weakness!

AJAXifing Templates and core features

Last post 3 Weeks ago

Lots of talk here trying to work out how to add ajax functionality to the magento site and surprise surprise, its not supported out of the box and rather complex for most people to try and work out. Instead, well:
Vivendo has a great extension and it’s very affordable!

Whoop-ee! Who’s Vivendo? Oh yeah, the people that write the magento core.

PCI DSS Compliance / Gift Certs / Logging of admin data

For processing cards onsite your cart system should comply to the PCI-DSS regulations. Magento community does not and should be used in situations where you hand payment to another system like paypal or do not store card data on site if using a gateway. The Enterprise (the $8000/yr) system is scheduled to be certified to comply to PCI-DSS soon. Partly because the Enterprise version includes data encryption which the community one does not.

While a Django system would not be “out of the box” PCI-DSS compliant we would be writing the code to store attributes etc and would be able to add an encryption layer (such as 3DES or blowfish etc) to that data. It should also be noted that only the Enterprise edition supports admin logging, gift certs and some nice “extras” such as store credit.

Also the enterprise version is the only one to allow for a proper “walled garden” of catalogues so for example having a section of the store for dealers and not general public orders is not possible with the stock-standard magento. Its also not possible to have sub-admins (could only add new products for example) either something which the django admin site has out of the box (and is easily customizable/extended).

Compare magento versions

Licensing

This is valid at the moment Magento Licesnsing:
As such, you must disclose any changes you make to the OSL 3.0-licensed copyrighted works whenever you distribute Magento or make your Magento store or software available over the web to a third party.

We would probably require a commercial license for magento (i.e. enterprise) if we did not want to re-release any changes to core. Judging from the way people talk about working with magento you will probably end up having to modify core at some point. A Django solution would be completely our own code and even hacking django core, its licensed under BSD meaning commercial works need not release derivative works back to the original source branch.

BSD License Terms
Due to the extremely minimal restrictions of BSD-style licenses, software released under such licenses can be freely modified and used in proprietary (i.e., commercial) software for which the source code is kept secret.

Payment Systems

Magento may or may not support the gateway that we want to use. They have support for a lot (most with an additional fee for the module, seeing a pattern here?):

Supported magento payment systems

Paypal, google checkout, sagepay (protx) are supported with free/paid extensions. The protx one (updated for sage pay) is in beta at the minute and there are 3 pages worth of reviews pointing out problems people are having with it. This is an extra reason for concern for me personally – having to rely on a load of extensions written all by different people, some charged, some not just to get the core functioning. What happens if there’s a bug in one of them?

This is a great extension and have Just upgraded to latest version and now it dont work? Using vewrsion 1.3.1.1 beta.

Fin

Ok, thats probably enough for now. Main point being that this system is customisable – for a fee. If they are mainly concerned about linking systems together then they really need to understand that the data can come as and how they want it. If they want XML of all their orders based on a SKU then we can provide that. If they have custom systems or expect data in such a way then we will fit output to that schema! It still feels like they think magento is this software like microsoft office that is going to somehow solve a load of problems off the bat.

One of the main things for me is that Magento is a dual licensed, essentially paid for system with a big consultant/community that work to push magento. Django is an open source BSD licensed framework built for developers to develop web applications on. Django does not offer paid-for licensed versions, its in django’s interest to have as much and as transparent documentation as possible. For Magento its not quite in their interest to have that same level of documentation, something which seems to be reflected when you read about developers experiences trying to extend it. Invariably it ends with a “magento expert” doing the work or the person struggling to fix it and ending up hacking core or finding some extension that just about does what they want and they live with it.

I know I’m going to end up having some “out of the ordinary” use cases when it comes to discounts, cart functioning and products themselves and the idea of working with magento does not thrill me. When you see posts on stackoverflowasking if theres any other documentation but the source code for the ORM then alarm bells ring. Especially when you’re used to the sort of quality & thorough documentation django provides

End rant! So yeah, I’m not the worlds biggest fan of magento. And I’ve not even touched on the actual quantity & scale of the db tables it installs on a fresh version. Magento fanatics please keep your “its brilliant & you are a knob” mail to yourself.