#summary How to get started with Melange
#labels Featured,Phase-Guidelines,Contents-Draft,Importance-Featured

<wiki:toc max_depth="1" />

= Prerequisites =

[MelangeIntro Melange] is a web application written in Python 2.7 that runs on
[http://code.google.com/appengine/ Google App Engine].

= TL;DR = 

You should read the text below, but if you want a quick refresher on how to get running, here's the TL;DR.

== Prerequisites == 

Building, testing, and running Melange depends on installation of a few system packages:
 * git
 * libxml2-dev
 * libxslt-dev
 * openjdk-7-jdk
 * python (version 2.7)
 * python-dev
 * python-lxml
 * virtualenv (version 1.10 or later)
 * xvfb

== Setting up ==

If you do not yet have a melange directory set up, run the following commands.

{{{
git clone http://code.google.com/p/soc/
cd soc
virtualenv venv
venv/bin/pip install -U lxml setuptools
venv/bin/python bootstrap.py
bin/buildout
bin/gen-app-yaml local-devel
bin/paver build
}}}

== Running locally ==

If you want to run melange locally, run the following command

{{{
thirdparty/google_appengine/dev_appserver.py build
}}}

== Creating an !AppEngine instance ==

If you do not already have an !AppEngine instance, follow these steps, otherwise you can skip this part.

 * Go to http://www.appspot.com and sign in with a Google Account
 * Click "Create an application"
 * Verify your account if prompted
 * In the "Create an Application" page, fill in an identifier (cannot be changed), you will need this later on when deploying
 * Fill in a title (can be changed later)
 * Set the "Authentication Options" on either "Open to all Google Accounts users (default)" or "Restricted to the following Google Apps domain".
 * Hit "Create Application"

== Deploying to your own instance ==

Assuming you have an !AppEngine instance called 'your-melange-instance'

{{{
scripts/gen_app_yaml.py -o devvin -f your-melange-instance &&
thirdparty/google_appengine/appcfg.py update build
}}}


== Running the test suite ==

If you want to run the test suite, run the following

{{{
bin/run-tests
}}}

This will run, by default, all types of tests. This is the same as running

{{{
bin/run-tests -t js,pyunit
}}}

So, using {{{ -t }}} switch it's possible to choose a list of comma-separated types of tests to run.

To run only pyunit tests:

{{{
bin/run-tests -t pyunit
}}}

To run only js tests:

{{{
bin/run-tests -t js
}}}

To run only functional tests(Headless):

{{{
bin/run-tests -t functional
}}}

To run only functional tests(In a browser):

{{{
bin/run-tests -t functional --browsers-gui
}}}

= Checking Out the Code =

To check out Melange source code run the following command:

Using Git repository:

{{{
git clone http://code.google.com/p/soc/
}}}

= Setting up your Client =
Change directories into the working directory (the one created by checking out the code). Google App Engine requires a file `app.yaml` in the application directory. For now, you can create a dummy `app.yaml` file by running:
{{{
scripts/gen_app_yaml.py local-devel -o devvin
}}}

= Running the Code Locally =
You can start the server on your computer by running:

{{{
thirdparty/google_appengine/dev_appserver.py build
}}}

You should now be able to see the running server at [http://localhost:8080/] .  Congrats!  You've got Melange running!

(If you want to run on a different port, you can use the `--port` argument.  Other arguments are described [http://code.google.com/appengine/docs/thedevwebserver.html here].)

(Since you are starting from scratch, you will probably want some sample data.
You can use Melange generated data to save yours fingers for code. First you need to login, visit [http://localhost:8080/login] for the same. After login visit [http://localhost:8080/seed_db] to have Melange generated data for you.)

= Running on and Deploying to App Engine's cluster =
If you want to run an instance of Melange on Google's App Engine infrastructure, there are a few more steps to make sure it deploys the right data to the right place.

== Set the application name ==
In `app/app.yaml` there's an application name that in the above steps was set to fake-application-name.  After registering an application on Google App Engine, use any text editor to change the line that starts with `application: fake-application-name` to instead read `application: <your-instance-name>`. You can also run the script again:

{{{
scripts/gen_app_yaml.py -o devvin -f proper-application-name 
}}}


== Building With Paver ==

Because
[http://code.google.com/p/soc/source/browse/#svn/trunk/app/tiny_mce Tiny MCE]
(and possibly some other third-party content and JavaScript incorporated
into Melange) is large,
[http://code.google.com/p/googleappengine/source/browse/#svn/trunk/google/appengine/ext/zipserve zipserve]
is used to combine this content into `.zip` archive files (to avoid
file-count limits in Google App Engine).  A side effect of this is that,
for Tiny MCE to work in the `textarea` controls on your local working copy,
launched with `thirdparty/google_appengine`, you still need to run a build script
to produce these <tt>.zip</tt> files at least once.

To do this, you will need to execute the `bin/paver` script, using the following command

{{{
bin/paver build
}}}

This script will create a `build/` directory at the same level as the
`app/` directory (and the `scripts/` directory) in the `hg` working copy.

== Deploying to Google App Engine ==

Once you have your application configured correctly and you have
executed the `paver` script found in the `bin` folder,
you are ready to deploy your Melange image to your specific Google
App Engine instance.

You can run the contents of the `build/` directory, in
order to test what would actually be uploaded by `appcfg.py upload`
to Google App Engine by using:

{{{
bin/gae_develop build
}}}

Once you are satisfied that `build/` contains everything needed, you can deploy
to your Google App Engine instance with:

{{{
thirdparty/google_appengine/appcfg.py update build
}}}

= Contributing =

There are a number of ways that you can contribute to the Melange project.

== Discussion ==

If you are interested in contributing to the Melange project, please join our
[MailingListsGuidelines#General_discussion_list general discussion mailing list].
Also,feel free to join us [IrcChannelGuidelines on IRC].


== Bug Hunting and Feature Requests ==

If you are (or were) a mentor or student participant in GSoC or GCI but do not have the time to contribute patches to Melange, we can still use your help.  While a great deal of code is already done, some parts of the web application are not entirely bug free yet, or feature complete.
Please consider writing up a feature request, or bug report. Please be precise when you file a [http://tinyurl.com/new-issue New Issue].

== Patches ==

[ContributionReviews#Patches Patches] to the code are also welcome,
[ContributionReviews#Suggested_patches_from_casual_contributors even from non-committers].

The code that is used for showing pages can be found in app/soc/modules/`<module_name>`/views.

= Troubleshooting =

==NoneTypeAttributeError==
{{{

Traceback (most recent call last):
  File "bootstrap.py", line 105, in <module>
    ws.find(pkg_resources.Requirement.parse(requirement)).location
AttributeError: 'NoneType' object has no attribute 'location'

}}}

[https://bugs.launchpad.net/zc.buildout/+bug/515398]

I was able to solve the problem by removing the python-pkg-resources package. I think the bootstrap script, being able to import pkg_resources, assumes that I have Distribute or Setuptools installed, which is not the case.


== AppConfigNotFoundError ==


{{{
soc-read-only$ thirdparty/google_appengine/dev_appserver.py app
Traceback (most recent call last):
  File "soc-read-only/thirdparty/google_appengine/google/appengine/tools/dev_appserver.py", line 2638, in LoadAppConfig
    raise AppConfigNotFoundError
google.appengine.tools.dev_appserver.AppConfigNotFoundError
}}}

This error means that it can not find a app.yaml file. Please check the following things,

 * You are running the command in the directory you checked out.
 * You have created the app.yaml from app.yaml.template in step [GettingStarted#Running_the_Code Running the Code].


== Socket.error ==  

{{{
File "/usr/lib/python2.5/BaseHTTPServer.py", line 101, in server_bind
    SocketServer.TCPServer.server_bind(self)
  File "/usr/lib/python2.5/SocketServer.py", line 341, in server_bind
    self.socket.bind(self.server_address)
  File "<string>", line 1, in bind
socket.error: (98, 'Address already in use')
}}}

This error means that some other program is running on port 8080 (the default port that appengine starts on). This can also happen if you try and run two copies of the program at once.

To run the program on a different port use the --port argument, for example
{{{
thirdparty/google_appengine/dev_appserver.py --port=9999 build
}}}

You will then need to connect to the correct port when browsing to the site. IE In the above example you would need to go to [http://localhost:9999/].

== BadValueError ==

{{{
File "/google_appengine/google/appengine/ext/db/__init__.py" in filter
  1482.       raise BadValueError('Filtering on lists is not supported')

Exception Type: BadValueError at /
Exception Value: Filtering on lists is not supported
}}}

This error means you are running a version of AppEngine that does not support the IN operator, please use the version of appengine that ships with Melange under /thirdparty/google_appengine, or update to a recent enough version (that is, >= 1.1.9).

== <type 'exceptions.ImportError'> ==  

Google App Engine SDK currently runs only on Python 2.5. If your system provides Python 2.6 or later as the default version of Python, you are likely to get the following warning as soon as you start the SDK by running 
{{{
thirdparty/google_appengine/dev_appserver.py build
}}}

{{{
thirdparty/google_appengine/google/appengine/tools/appcfg.py:40: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
INFO     2009-04-29 19:49:06,517 appengine_rpc.py] Server: appengine.google.com
INFO     2009-04-29 19:49:06,529 appcfg.py] Checking for updates to the SDK.
thirdparty/google_appengine/google/appengine/api/datastore_types.py:555: DeprecationWarning: object.__init__() takes no parameters
  super(Email, self).__init__(self, email)
thirdparty/google_appengine/google/appengine/api/datastore_types.py:745: DeprecationWarning: object.__init__() takes no parameters
  super(PhoneNumber, self).__init__(self, phone)
thirdparty/google_appengine/google/appengine/api/datastore_types.py:532: DeprecationWarning: object.__init__() takes no parameters
  super(Link, self).__init__(self, link)
}}}

And when you try to access a page using any URL, you get this exception
{{{
  File "thirdparty/google_appengine/google/appengine/tools/dev_appserver.py", line 996, in decorate
    return func(self, *args, **kwargs)
  File "thirdparty/google_appengine/google/appengine/tools/dev_appserver.py", line 1444, in LoadModuleRestricted
    description)
  File "/usr/lib/python2.6/multiprocessing/__init__.py", line 83, in <module>
    import _multiprocessing
ImportError: No module named _multiprocessing
}}}

The workaround for this is to install Python 2.5 on your machine. Note that you should _only_ do this if you get an exception when opening a page, if you only get DeprecationWarnings you can safely ignore those. It will be then be accessible by using python2.5 on the command line. Then open thirdparty/google_appengine/dev_appserver.py in your favorite text editor and replace the first line, 
{{{
#!/usr/bin/env python
}}}

with 
{{{
#!/usr/bin/env python2.5
}}}

Save the changes in the file, close it and restart the GAE development server.

Depending on the operating system (Mac OS X, different linux distributions) you just might be able to create symbolic link `python` for `python2.5` file that way you don't have to modify dev_appserver.py file.


== <class 'django.core.exceptions.ImproperlyConfigured'> ==  

{{{
Error importing middleware app_profiler.app_profiler: "No module named pstats"
args = ('Error importing middleware app_profiler.app_profiler: "No module named pstats"',)
message = 'Error importing middleware app_profiler.app_profiler: "No module named pstats"'
}}}

This error means you're missing _pstats_ Python module. It happens at least on Ubuntu Jaunty. This module is part of _python-profiler_ package. So to install it you can open a console and write:

{{{
sudo apt-get install python-profiler
}}}

== setuptools-0.6c11-py2.6.egg-info: Permission denied ==

{{{
$ python bootstrap.py 
install_dir /tmp/tmp9byusF
Getting distribution for 'distribute'.
install_dir /home/sverre/code/Melange/eggs/tmp6eeuMe
Before install bootstrap.
Scanning installed packages
Setuptools installation detected at /usr/lib/python2.6/dist-packages
Non-egg installation
Removing elements out of the way...
Already patched.
/usr/lib/python2.6/dist-packages/setuptools.egg-info already patched.
After install bootstrap.
Creating /usr/local/lib/python2.6/dist-packages/setuptools-0.6c11-py2.6.egg-info
  Getting distribution for 'distribute'.
Error: Couldn't install: distribute 0.6.15
}}}

Related: https://bugs.launchpad.net/zc.buildout/+bug/656303
There's an easy workaround:

{{{
sudo easy_install distribute==0.6.15
}}}

== ImportError: No module named _sqlite3 ==

{{{
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.5/sqlite3/__init__.py", line 24, in <module>
    from dbapi2 import *
  File "/usr/local/lib/python2.5/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: No module named _sqlite3
}}}

This happens when you've manually compiled python 2.5, you should consider instead using the version of python that comes with your OS. If not, compile python 2.5 with sqlite support. See also [http://stackoverflow.com/questions/1210664/no-module-named-sqlite3 this article] on stackoverflow.

== ImportError: No 'check_packages' attribute ==

{{{
ImportError: <module 'setuptools.dist' from ...> has no 'check_packages' attribute
}}}

If your bin/buildout fails with this error and you run bootstrap.py with python 2.5, try running it again with python2.6 instead.

== pkg_resources.VersionConflict: (setuptools 0.6c11 (/usr/lib/python2.7/dist-packages), Requirement.parse('setuptools>=0.7')) ==

{{{
$ python bootstrap.py 
Traceback (most recent call last):  File "bootstrap.py", line 159, in <module>    ws.require(requirement)  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 686, in require    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 588, in resolve    raise VersionConflict(dist,req) # XXX put more info herepkg_resources.VersionConflict: (setuptools 0.6c11 (/usr/lib/python2.7/dist-packages), Requirement.parse('setuptools>=0.7'))
}}}

Your version of setuptools is too old, the easiest way to fix this is to run:
{{{
sudo easy_install -U setuptools
sudo easy_install -U virtualenv # optional
}}}