.. _appinstall: .. _install: Installing PyAMS application ============================ **PyAMS** default installation is based on `Buildout `_ utility. .. tip:: Take a look at virtualenv_. This tool provide isolated Python environments, which are more practical than installing packages systemwide. They also allow installing packages without administrator privileges. .. _virtualenv: https://virtualenv.pypa.io/en/stable/ Current PyAMS version is based and validated for **Python 3.5**; your Python environment must also include a **C compiler** as well as development headers for Python, *libjpeg*, *libpng*, *libfreetype*, *libxml2*, *libxslt* and eventually *libldap*, *libpq*, *libffi*, *libgdal* or *libzmq*. **Cython** compiler can also be useful to optimize several packages. .. note:: PyAMS default components configuration also pre-suppose that the following external services are available. * Mandatory services: - a **ZODB shared server** (see :ref:`zodb` before). Several background processes needing a concurrent access to ZODB are started by PyAMS main process. Three ZODB storages are already provided through PyAMS: ZEO, RelStorage or Newt.db. - a **Memcached** or **Redis*** server, to store sessions and cache * Optional services: - an **LDAP server** for authentication - an **ElasticSearch server** for full text indexing (see *PyAMS_content_es* package) - a **WebSockets server** using AsyncIO. This is used to manage notifications (see *PyAMS_notify* and *PyAMS_notify_ws* packages). An *out of the box* environment can be built using *pyams_notify* scaffold. Creating initial buildout ------------------------- PyAMS provides a new Pyramid scaffold, called *pyams*, generated via a *cookiecutter* template. A simple option to install PyAMS is to create a buildout environment including *Pyramid* and all *PyAMS* packages: .. code-block:: bash $ mkdir /var/local/ $ pip3 install virtualenv $ virtualenv --python=python3.5 env $ cd env $ . bin/activate (env) $ pip3.5 install cookiecutter (env) $ cookiecutter hg+http://hg.ztfy.org/pyams/scaffolds/pyams *CookieCutter* will ask you for a small set of input variables that you can change or not (**parameter** [*default value*]:) - **pyams_release** [*latest*] : version of PyAMS configuration file to use. "latest" will point to last release; you can also choose to point to a given release ("0.1.4" for example) - **project_name** [*PyAMS*]: current environment name in "human form" - **project_slug**: [*pyams*] "technical" package name, based on project name - **virtual_hostname** [*pyams.mydomain.com*]: Apache virtual-host name - **webapp_name** [*webapps*]: web application package name - **webapp_port** [*6543*]: TCP/IP port to use when running application outside Apache. - **eggs_directory** [*eggs*]: relative or absolute path to directory containing downloaded eggs; this directory can be shared with other projects. - **logs_directory** [*/var/log*]: absolute path to directory containing Apache's log files - **run_user** [*www-data*]: user name under which Apache process will run - **run_group** [*www-data*]: group name under which Apache process will run - **beaker_backend** : name of Beaker backend to use to store sessions and cache data ("redis" as default) - **beaker_server** [*127.0.0.1:6379*]: IP address and port of Beaker backend server - **db_type** [*zeo*]: ZODB database storage; available options include ZEO, RelStorage and NewtDB - **db_host** [*127.0.0.1*]: IP address of database server ("127.0.0.1" as default); WARNING: database server installation is not part of application installation; another "zeo_server" cookiecutter recipe is available for ZEO - **db_port** [*8100*]: listening port of database server ("8100" is given as default for ZEO) - **db_name** [*pyams*]: database or ZEO storage name to use - **db_username**: database user name - **db_password**: database password - **zeo_realm**: ZEO authentication realm - **blobs_dir** [*/var/db/blobs*]: local directory to use to store cache of ZODB blobs; cache size is limited to 10GB as default - **use_postgresql**: specify if PostgreSQL access is required; if so, please check that PostgreSQL development files are available to compile PsycoPG2 extension - **use_oracle**: specify if Oracle access is required; if so, please check that Oracle development files are available to compile cx_Oracle extension, and that ORACLE_HOME environment variable is correctly defined (see below) - **use_ldap**: specify if LDAP access will be required for authentication - **use_elasticsearch**: specify if an ElasticSearch server will be used for indexation - **elasticsearch_server**: URL used to access Elasticsearch server ("http://127.0.0.1:9200" as default); this URL can include login and password ("http://login:password@127.0.0.1:9200"), if required... - **elasticsearch_index**: name of Elasticsearch index to use ("pyams" as default) - **create_elasticsearch_index**: specify if Elasticsearch index should be created after installation is complete - **define_elasticsearch_mappings** : specify if Elasticsearch mappings should be defined after installation is complete - **smtp_server**: DNS name of SMTP server ("localhost" as default) - **smtp_server_name**: "human" name given to SMTP server ("pyams" as default) - **pyams_scheduler** [*127.0.0.1:5555*]: TCP/IP address and port to use to access PyAMS tasks scheduler process (see :ref:`pyams_scheduler`) - **start_scheduler** [*True*]: boolean value to indicate if scheduler process is started by this application instance - **pyams_medias_converter** [*127.0.0.1:5556*]: TCP/IP address and port to use to access PyAMS medias converter process (see :ref:`pyams_media`) - **start_medias_converter** [*True*]: boolean value to indicate if medias converter process is started by this application instance - **pyams_es_indexer** [*127.0.0.1:5557*] : TCP/IP address and port to use to access PyAMS Elasticsearch indexer process (see :ref:`pyams_content_es`) - **start_es_indexer** boolean value to indicate if Elasticsearch indexer process is started by this application instance - **use_notifications**: specify if PyAMS notifications services are to be used (see :ref:`pyams_notify`) - **pyams_ws_notify** [*127.0.0.1:8081*]: TCP/IP address and port of PyAMS websockets server managing notifications service - **lexicon_languages**: NLTK lexicon languages to use ("en:english fr:french" as default) - **extension_package**: name of a PyAMS extension package to include in environment configuration - **need_pyams_gis**: specify if PyAMS GIS features are to be used by given extension package; if so, please check that *libgdal* development files are available; on Debian (and maybe others), you have to specify environment variables (see below). .. tip:: 1. Setting any option to empty string will keep the default proposed value. 2. Boolean values accept "true", "yes", "on" or "1" (in any case) as *true* values, and anything else as *false*; You can then check, and eventually update, the proposed Buildout configuration file *buildout.cfg*, to add or remove packages or update settings to your needs. This last operation can be quite long, as many packages have to downloaded, compiled and installed in the virtual environment. If you encounter any compile error, just install the required dependencies and restart the buildout. Some dependencies can require the definition of custom environment variables before running *buildout*, like: - for *libgdal*, which is required by **PyAMS_gis** package, use: .. code-block:: bash (env) $ export C_INCLUDE_PATH=/usr/include/gdal (env) $ export CPLUS_INCLUDE_PATH=/usr/include/gdal .. WARNING:: You have to check also that your *libgdal* release is matching "GDAL" release given in PyAMS configuration file (actually 2.1.0). - for *cx_Oracle*, which is required if you use Oracle database connections, use: .. code-block:: bash (env) $ export ORACLE_HOME=/usr/lib/oracle/12.1/client64 These examples are given for Debian GNU/Linux. You may have to adapt configuration based on your own Linux distribution and packages versions. **Then finalize Bootstrap initialization:** .. code-block:: bash (env) $ python3.5 bootstrap.py (env) $ ./bin/buildout .. seealso:: `zc.buildout `_ is a powerful tool for creating repeatable builds of a given software configuration and environment. Environment settings -------------------- The project generated from *pyams* scaffold is based on default Pyramid's *zodb* scaffold, but it adds: - a custom application factory, in the *webapp* directory (see :ref:`site`) - a set of directories to store runtime data, in the *var* directory; each directory contains a *README.txt* file which should be self-explanatory to indicate what this directory should contain, including a ZEO cache - a set of configuration files, in the *etc* directory; here are standard *development.ini* and *production.ini* configuration files, a ZODB configuration files (*zodb-zeo.conf*) for a ZEO client storage and two Apache configurations (for Apache 2.2 and 2.4) using *mod_wsgi*. Once the project have been created from the scaffold, you are free to update all the configuration files. If you need to add packages to the environment, you have to add them to the *buildout.cfg* file **AND** to the INI file (in the *pyramid.includes* section) before running the *buildout* another time; don't forget to add the requested version at the end of *buildout.cfg* file, as Buildout is not configured by default to automatically download the last release of a given unknown package. *development.ini* and *production.ini* files contain many commented directives related to PyAMS components. Read and update them carefully before initializing your application database!