mirror of
https://github.com/Fishwaldo/huginn.git
synced 2025-03-21 06:11:39 +00:00
Merge pull request #1204 from dsander/docker
Use production env in Docker image, add single process Docker image
This commit is contained in:
Dominik Sander
commit
89a824cffa
19 changed files with 559 additions and 245 deletions
|
|
@ -1,36 +0,0 @@
|
|||
FROM ubuntu:14.04
|
||||
MAINTAINER Andrew Cantino
|
||||
|
||||
ENV DEBIAN_FRONTEND noninteractive
|
||||
RUN apt-get update && \
|
||||
apt-get install -y software-properties-common && \
|
||||
add-apt-repository -y ppa:git-core/ppa && \
|
||||
add-apt-repository -y ppa:brightbox/ruby-ng && \
|
||||
apt-get update && \
|
||||
apt-get install -y build-essential checkinstall postgresql-client \
|
||||
git-core mysql-server redis-server python2.7 python-docutils \
|
||||
libmysqlclient-dev libpq-dev zlib1g-dev libyaml-dev libssl-dev \
|
||||
libgdbm-dev libreadline-dev libncurses5-dev libffi-dev \
|
||||
libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev \
|
||||
graphviz libgraphviz-dev \
|
||||
ruby2.1 ruby2.1-dev supervisor python-pip && \
|
||||
gem install --no-ri --no-rdoc bundler && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN pip install supervisor-stdout
|
||||
|
||||
ADD scripts/setup /scripts/setup
|
||||
RUN chmod 755 /scripts/setup
|
||||
RUN /scripts/setup
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
ADD scripts/init /scripts/init
|
||||
RUN chmod 755 /scripts/init
|
||||
|
||||
VOLUME /var/lib/mysql
|
||||
|
||||
EXPOSE 3000
|
||||
|
||||
CMD ["/scripts/init"]
|
||||
|
||||
154
docker/README.md
154
docker/README.md
|
|
@ -1,152 +1,12 @@
|
|||
Huginn for docker with multiple container linkage
|
||||
=================================================
|
||||
Huginn Docker images
|
||||
====================
|
||||
|
||||
This image runs a linkable [Huginn](https://github.com/cantino/huginn) instance.
|
||||
Huginn is packaged in two docker images.
|
||||
|
||||
There is an automated build repository on docker hub for [cantino/huginn](https://registry.hub.docker.com/builds/github/cantino/huginn/).
|
||||
#### `cantino/huginn` multiple process image
|
||||
|
||||
This was patterned after [sameersbn/gitlab](https://registry.hub.docker.com/u/sameersbn/gitlab) by [ianblenke/huginn](http://github.com/ianblenke/huginn), and imported here for official generation of a docker hub auto-build image.
|
||||
|
||||
The scripts/init script generates a .env file containing the variables as passed as per normal Huginn documentation.
|
||||
The same environment variables that would be used for Heroku PaaS deployment are used by this script.
|
||||
|
||||
The scripts/init script is aware of mysql and postgres linked containers through the environment variables:
|
||||
|
||||
MYSQL_PORT_3306_TCP_ADDR
|
||||
MYSQL_PORT_3306_TCP_PORT
|
||||
|
||||
and
|
||||
|
||||
POSTGRESQL_PORT_5432_TCP_ADDR
|
||||
POSTGRESQL_PORT_5432_TCP_PORT
|
||||
|
||||
Its recommended to use an image that allows you to create a database via environmental variables at docker run, like `paintedfox / postgresql` or `centurylink / mysql`, so the db is populated when this script runs.
|
||||
|
||||
If you do not link a database container, a built-in mysql database will be started.
|
||||
There is an exported docker volume of /var/lib/mysql to allow persistence of that mysql database.
|
||||
|
||||
Additionally, the database variables may be overridden from the above as per the standard Huginn documentation:
|
||||
|
||||
HUGINN_DATABASE_ADAPTER #(must be either 'postgres' or 'mysql2')
|
||||
HUGINN_DATABASE_HOST
|
||||
HUGINN_DATABASE_PORT
|
||||
|
||||
This script will run database migrations (rake db:migrate) which should be idempotent.
|
||||
|
||||
It will also seed the database (rake db:seed) unless this is defined:
|
||||
|
||||
DO_NOT_SEED
|
||||
|
||||
This same seeding initially defines the "admin" user with a default password of "password" as per the standard Huginn documentation.
|
||||
|
||||
If you do not wish to have the default 6 agents, you will want to set the above environment variable after your initially deploy, otherwise they will be added automatically the next time a container pointing at the database is spun up.
|
||||
|
||||
The CMD launches Huginn via the scripts/init script. This may become the ENTRYPOINT later. It does take under a minute for Huginn to come up. Use environmental variables that match your DB's creds to ensure it works.
|
||||
|
||||
## Usage
|
||||
|
||||
Simple stand-alone usage:
|
||||
|
||||
docker run -it -p 3000:3000 cantino/huginn
|
||||
|
||||
To link to another mysql container, for example:
|
||||
|
||||
docker run --rm --name huginn_mysql \
|
||||
-e MYSQL_DATABASE=huginn \
|
||||
-e MYSQL_USER=huginn \
|
||||
-e MYSQL_PASSWORD=somethingsecret \
|
||||
-e MYSQL_ROOT_PASSWORD=somethingevenmoresecret \
|
||||
mysql
|
||||
docker run --rm --name huginn \
|
||||
--link huginn_mysql:mysql \
|
||||
-p 3000:3000 \
|
||||
-e HUGINN_DATABASE_NAME=huginn \
|
||||
-e HUGINN_DATABASE_USERNAME=huginn \
|
||||
-e HUGINN_DATABASE_PASSWORD=somethingsecret \
|
||||
cantino/huginn
|
||||
|
||||
To link to another container named 'postgres':
|
||||
|
||||
docker run --name huginn_postgres \
|
||||
-e POSTGRES_PASSWORD=mysecretpassword \
|
||||
-e POSTGRES_USER=huginn -d postgres
|
||||
docker run -it --link huginn_postgres:postgres --rm postgres \
|
||||
sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U huginn -c "create database huginn_development;"'
|
||||
docker run --rm --name huginn \
|
||||
--link huginn_postgres:postgres \
|
||||
-p 3000:3000 \
|
||||
-e HUGINN_DATABASE_USERNAME=huginn \
|
||||
-e HUGINN_DATABASE_PASSWORD=mysecretpassword \
|
||||
-e HUGINN_DATABASE_ADAPTER=postgresql \
|
||||
cantino/huginn
|
||||
|
||||
The `docker/` folder also has a `docker-compose.yml` that allows for a sample database formation with a data volume container:
|
||||
|
||||
cd docker ; docker-compose up
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Other Huginn 12factored environment variables of note, as generated and put into the .env file as per Huginn documentation,
|
||||
with an additional `HUGINN_` prefix to the variable.
|
||||
|
||||
These are:
|
||||
|
||||
HUGINN_APP_SECRET_TOKEN
|
||||
HUGINN_DOMAIN
|
||||
HUGINN_ASSET_HOST
|
||||
HUGINN_DATABASE_ADAPTER
|
||||
HUGINN_DATABASE_ENCODING
|
||||
HUGINN_DATABASE_RECONNECT
|
||||
HUGINN_DATABASE_NAME
|
||||
HUGINN_DATABASE_POOL
|
||||
HUGINN_DATABASE_USERNAME
|
||||
HUGINN_DATABASE_PASSWORD
|
||||
HUGINN_DATABASE_HOST
|
||||
HUGINN_DATABASE_PORT
|
||||
HUGINN_DATABASE_SOCKET
|
||||
HUGINN_RAILS_ENV
|
||||
HUGINN_FORCE_SSL
|
||||
HUGINN_INVITATION_CODE
|
||||
HUGINN_SMTP_DOMAIN
|
||||
HUGINN_SMTP_USER_NAME
|
||||
HUGINN_SMTP_PASSWORD
|
||||
HUGINN_SMTP_SERVER
|
||||
HUGINN_SMTP_PORT
|
||||
HUGINN_SMTP_AUTHENTICATION
|
||||
HUGINN_SMTP_ENABLE_STARTTLS_AUTO
|
||||
HUGINN_EMAIL_FROM_ADDRESS
|
||||
HUGINN_AGENT_LOG_LENGTH
|
||||
HUGINN_TWITTER_OAUTH_KEY
|
||||
HUGINN_TWITTER_OAUTH_SECRET
|
||||
HUGINN_THIRTY_SEVEN_SIGNALS_OAUTH_KEY
|
||||
HUGINN_THIRTY_SEVEN_SIGNALS_OAUTH_SECRET
|
||||
HUGINN_AWS_ACCESS_KEY_ID
|
||||
HUGINN_AWS_ACCESS_KEY
|
||||
HUGINN_AWS_SANDBOX
|
||||
HUGINN_FARADAY_HTTP_BACKEND
|
||||
HUGINN_DEFAULT_HTTP_USER_AGENT
|
||||
HUGINN_ALLOW_JSONPATH_EVAL
|
||||
HUGINN_ENABLE_INSECURE_AGENTS
|
||||
HUGGIN_ENABLE_SECOND_PRECISION_SCHEDULE
|
||||
HUGINN_USE_GRAPHVIZ_DOT
|
||||
HUGINN_TIMEZONE
|
||||
HUGGIN_FAILED_JOBS_TO_KEEP
|
||||
|
||||
|
||||
The above environment variables will override the defaults. The defaults are read from the [.env.example](https://github.com/cantino/huginn/blob/master/.env.example) file.
|
||||
|
||||
For variables in the .env.example that are commented out, the default is to not include that variable in the generated .env file.
|
||||
|
||||
## Building on your own
|
||||
|
||||
You don't need to do this on your own, because there is an [automated build](https://registry.hub.docker.com/u/cantino/huginn/) for this repository, but if you really want:
|
||||
|
||||
docker build --rm=true --tag={yourname}/huginn .
|
||||
|
||||
## Source
|
||||
|
||||
The source is [available on GitHub](https://github.com/cantino/huginn/).
|
||||
|
||||
Please feel free to submit pull requests and/or fork at your leisure.
|
||||
This image runs all processes needed by Huginn in one container, when not database is linked it will also start MySQL internally. It is great to try huginn without having to set up anything, however maintenance and backups can be difficult.
|
||||
|
||||
#### `cantino/huginn-single-process` multiple container image
|
||||
|
||||
This image runs just one process per container and thus needs at least two container to be started, one for the Huginn application server and one for the threaded background worker. It is also possible to every background worker in a separate container to improve the performance. See [the PostgreSQL docker-compose configuration](single-process/postgres.yml) for an example.
|
||||
|
|
|
|||
16
docker/multi-process/Dockerfile
Normal file
16
docker/multi-process/Dockerfile
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
FROM cantino/huginn-single-process:latest
|
||||
MAINTAINER Andrew Cantino
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
ADD scripts/standalone-packages /scripts/standalone-packages
|
||||
RUN /scripts/standalone-packages
|
||||
|
||||
ADD scripts/init /scripts/init
|
||||
|
||||
VOLUME /var/lib/mysql
|
||||
|
||||
EXPOSE 3000
|
||||
|
||||
CMD ["/scripts/init"]
|
||||
|
||||
103
docker/multi-process/README.md
Normal file
103
docker/multi-process/README.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
Huginn for docker with multiple container linkage
|
||||
=================================================
|
||||
|
||||
This image runs a linkable [Huginn](https://github.com/cantino/huginn) instance.
|
||||
|
||||
There is an automated build repository on docker hub for [cantino/huginn](https://registry.hub.docker.com/builds/github/cantino/huginn/).
|
||||
|
||||
This was patterned after [sameersbn/gitlab](https://registry.hub.docker.com/u/sameersbn/gitlab) by [ianblenke/huginn](http://github.com/ianblenke/huginn), and imported here for official generation of a docker hub auto-build image.
|
||||
|
||||
The scripts/init script generates a .env file containing the variables as passed as per normal Huginn documentation.
|
||||
The same environment variables that would be used for Heroku PaaS deployment are used by this script.
|
||||
|
||||
The scripts/init script is aware of mysql and postgres linked containers through the environment variables:
|
||||
|
||||
MYSQL_PORT_3306_TCP_ADDR
|
||||
MYSQL_PORT_3306_TCP_PORT
|
||||
|
||||
and
|
||||
|
||||
POSTGRESQL_PORT_5432_TCP_ADDR
|
||||
POSTGRESQL_PORT_5432_TCP_PORT
|
||||
|
||||
Its recommended to use an image that allows you to create a database via environmental variables at docker run, like `paintedfox / postgresql` or `centurylink / mysql`, so the db is populated when this script runs.
|
||||
|
||||
If you do not link a database container, a built-in mysql database will be started.
|
||||
There is an exported docker volume of /var/lib/mysql to allow persistence of that mysql database.
|
||||
|
||||
Additionally, the database variables may be overridden from the above as per the standard Huginn documentation:
|
||||
|
||||
DATABASE_ADAPTER #(must be either 'postgres' or 'mysql2')
|
||||
DATABASE_HOST
|
||||
DATABASE_PORT
|
||||
|
||||
This script will run database migrations (rake db:migrate) which should be idempotent.
|
||||
|
||||
It will also seed the database (rake db:seed) unless this is defined:
|
||||
|
||||
DO_NOT_SEED
|
||||
|
||||
This same seeding initially defines the "admin" user with a default password of "password" as per the standard Huginn documentation.
|
||||
|
||||
If you do not wish to have the default 6 agents, you will want to set the above environment variable after your initially deploy, otherwise they will be added automatically the next time a container pointing at the database is spun up.
|
||||
|
||||
The CMD launches Huginn via the scripts/init script. This may become the ENTRYPOINT later. It does take under a minute for Huginn to come up. Use environmental variables that match your DB's creds to ensure it works.
|
||||
|
||||
## Usage
|
||||
|
||||
Simple stand-alone usage:
|
||||
|
||||
docker run -it -p 3000:3000 cantino/huginn
|
||||
|
||||
To link to another mysql container, for example:
|
||||
|
||||
docker run --rm --name huginn_mysql \
|
||||
-e MYSQL_DATABASE=huginn \
|
||||
-e MYSQL_USER=huginn \
|
||||
-e MYSQL_PASSWORD=somethingsecret \
|
||||
-e MYSQL_ROOT_PASSWORD=somethingevenmoresecret \
|
||||
mysql
|
||||
docker run --rm --name huginn \
|
||||
--link huginn_mysql:mysql \
|
||||
-p 3000:3000 \
|
||||
-e HUGINN_DATABASE_NAME=huginn \
|
||||
-e HUGINN_DATABASE_USERNAME=huginn \
|
||||
-e HUGINN_DATABASE_PASSWORD=somethingsecret \
|
||||
cantino/huginn
|
||||
|
||||
To link to another container named 'postgres':
|
||||
|
||||
docker run --name huginn_postgres \
|
||||
-e POSTGRES_PASSWORD=mysecretpassword \
|
||||
-e POSTGRES_USER=huginn -d postgres
|
||||
docker run --rm --name huginn \
|
||||
--link huginn_postgres:postgres \
|
||||
-p 3000:3000 \
|
||||
-e HUGINN_DATABASE_USERNAME=huginn \
|
||||
-e HUGINN_DATABASE_PASSWORD=mysecretpassword \
|
||||
-e HUGINN_DATABASE_ADAPTER=postgresql \
|
||||
cantino/huginn
|
||||
|
||||
The `docker/multi-process` folder also has a `docker-compose.yml` that allows for a sample database formation with a data volume container:
|
||||
|
||||
cd docker/multi-process ; docker-compose up
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Other Huginn 12factored environment variables of note, as generated and put into the .env file as per Huginn documentation. All variables of the [.env.example](https://github.com/cantino/huginn/blob/master/.env.example) can be used to override the defaults which a read from the current `.env.example`.
|
||||
|
||||
For variables in the .env.example that are commented out, the default is to not include that variable in the generated .env file.
|
||||
|
||||
## Building on your own
|
||||
|
||||
You don't need to do this on your own, because there is an [automated build](https://registry.hub.docker.com/u/cantino/huginn/) for this repository, but if you really want:
|
||||
|
||||
docker build --rm=true --tag={yourname}/huginn .
|
||||
|
||||
## Source
|
||||
|
||||
The source is [available on GitHub](https://github.com/cantino/huginn/docker/multi-process/).
|
||||
|
||||
Please feel free to submit pull requests and/or fork at your leisure.
|
||||
|
||||
|
||||
|
|
@ -3,9 +3,6 @@ set -e
|
|||
|
||||
cd /app
|
||||
|
||||
# Default to the environment variable values set in .env.example
|
||||
source /app/.env.example
|
||||
|
||||
# Cleanup any leftover pid file
|
||||
if [ -f /app/tmp/pids/server.pid ]; then
|
||||
rm /app/tmp/pids/server.pid
|
||||
|
|
@ -15,28 +12,42 @@ fi
|
|||
# requires that the mysql or postgresql containers have exposed
|
||||
# port 3306 and 5432 respectively.
|
||||
if [ -n "${MYSQL_PORT_3306_TCP_ADDR}" ]; then
|
||||
HUGINN_DATABASE_ADAPTER=${HUGINN_DATABASE_ADAPTER:-mysql2}
|
||||
HUGINN_DATABASE_HOST=${HUGINN_DATABASE_HOST:-${MYSQL_PORT_3306_TCP_ADDR}}
|
||||
HUGINN_DATABASE_PORT=${HUGINN_DATABASE_PORT:-${MYSQL_PORT_3306_TCP_PORT}}
|
||||
DATABASE_ADAPTER=${DATABASE_ADAPTER:-mysql2}
|
||||
DATABASE_HOST=${DATABASE_HOST:-${MYSQL_PORT_3306_TCP_ADDR}}
|
||||
DATABASE_PORT=${DATABASE_PORT:-${MYSQL_PORT_3306_TCP_PORT}}
|
||||
DATABASE_ENCODING=${DATABASE_ENCODING:-utf8mb4}
|
||||
elif [ -n "${POSTGRES_PORT_5432_TCP_ADDR}" ]; then
|
||||
HUGINN_DATABASE_ADAPTER=${HUGINN_DATABASE_ADAPTER:-postgresql}
|
||||
HUGINN_DATABASE_HOST=${HUGINN_DATABASE_HOST:-${POSTGRES_PORT_5432_TCP_ADDR}}
|
||||
HUGINN_DATABASE_PORT=${HUGINN_DATABASE_PORT:-${POSTGRES_PORT_5432_TCP_PORT}}
|
||||
DATABASE_ADAPTER=${DATABASE_ADAPTER:-postgresql}
|
||||
DATABASE_HOST=${DATABASE_HOST:-${POSTGRES_PORT_5432_TCP_ADDR}}
|
||||
DATABASE_PORT=${DATABASE_PORT:-${POSTGRES_PORT_5432_TCP_PORT}}
|
||||
DATABASE_ENCODING=utf8
|
||||
else
|
||||
START_MYSQL=${START_MYSQL:-true}
|
||||
fi
|
||||
|
||||
grep = /app/.env.example | sed -e 's/^#\([^ ]\)/\1/' | grep -v -e '^#' | cut -d= -f1 | \
|
||||
while read var ; do
|
||||
eval "echo \"$var=\\\"\${HUGINN_$var:-\$$var}\\\"\""
|
||||
USE_GRAPHVIZ_DOT=${HUGINN_USE_GRAPHVIZ_DOT:-${USE_GRAPHVIZ_DOT:-dot}}
|
||||
DATABASE_HOST=${HUGINN_DATABASE_HOST:-${DATABASE_HOST:-localhost}}
|
||||
DATABASE_PORT=${HUGINN_DATABASE_PORT:-${DATABASE_PORT}}
|
||||
DATABASE_ENCODING=${HUGINN_DATABASE_ENCODING:-${DATABASE_ENCODING}}
|
||||
DATABASE_PASSWORD=${HUGINN_DATABASE_PASSWORD:-${DATABASE_PASSWORD:-password}}
|
||||
DATABASE_NAME=${HUGINN_DATABASE_NAME:-${DATABASE_NAME:-huginn_production}}
|
||||
RAILS_ENV=${HUGINN_RAILS_ENV:-${RAILS_ENV:-production}}
|
||||
|
||||
IFS="="
|
||||
grep = /app/.env.example | sed -e 's/^#\([^ ]\)/\1/' | grep -v -e '^#' | \
|
||||
while read var value ; do
|
||||
eval "echo \"$var=\${$var:-\${HUGINN_$var-\$value}}\""
|
||||
done | grep -v -e ^= > /app/.env
|
||||
|
||||
eval "echo RAILS_ENV=${RAILS_ENV}" >> .env
|
||||
eval "echo START_MYSQL=${START_MYSQL}" >> .env
|
||||
echo "ON_HEROKU=true" >> .env
|
||||
echo "RAILS_SERVE_STATIC_FILES=true" >> .env
|
||||
|
||||
chmod ugo+r /app/.env
|
||||
source /app/.env
|
||||
sudo -u huginn -H bundle install --deployment --without test
|
||||
|
||||
DATABASE_HOST=${HUGINN_DATABASE_HOST:-${DATABASE_HOST:-localhost}}
|
||||
DATABASE_ENCODING=${HUGINN_DATABASE_ENCODING:-${DATABASE_ENCODING}}
|
||||
USE_GRAPHVIZ_DOT=${HUGINN_USE_GRAPHVIZ_DOT:-${USE_GRAPHVIZ_DOT}}
|
||||
|
||||
# use default port number if it is still not set
|
||||
case "${DATABASE_ADAPTER}" in
|
||||
mysql2) DATABASE_PORT=${DATABASE_PORT:-3306} ;;
|
||||
|
|
@ -71,7 +82,8 @@ source /app/.env
|
|||
echo DATABASE_HOST=\${DATABASE_HOST}
|
||||
|
||||
# start mysql server if \${DATABASE_HOST} is the .env.example default
|
||||
if [ "\${DATABASE_HOST}" = "your-domain-here.com" -o "\${DATABASE_HOST}" = "" ]; then
|
||||
if [ "\${START_MYSQL}" = "true" ]; then
|
||||
echo "DATABASE_SOCKET=/var/run/mysqld/mysqld.sock" >> .env
|
||||
if [ "\${DATABASE_ADAPTER}" = "postgresql" ]; then
|
||||
echo "DATABASE_ADAPTER 'postgresql' is not supported internally. Please provide DATABASE_HOST."
|
||||
exit 1
|
||||
|
|
@ -106,7 +118,7 @@ EOF
|
|||
|
||||
# wait for mysql server to start (max 120 seconds)
|
||||
timeout=120
|
||||
while ! mysqladmin -u\${DATABASE_USERNAME:-root} \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD} status >/dev/null 2>&1
|
||||
while ! mysqladmin -u root status >/dev/null 2>&1
|
||||
do
|
||||
(( timeout = timeout - 1 ))
|
||||
if [ \$timeout -eq 0 ]; then
|
||||
|
|
@ -118,9 +130,7 @@ EOF
|
|||
done
|
||||
|
||||
if ! echo "USE \${DATABASE_NAME}" | mysql -u\${DATABASE_USERNAME:-root} \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD} >/dev/null 2>&1; then
|
||||
DB_INIT="yes"
|
||||
echo "CREATE DATABASE IF NOT EXISTS \\\`\${DATABASE_NAME}\\\` DEFAULT CHARACTER SET \\\`utf8\\\` COLLATE \\\`utf8_unicode_ci\\\`;" | mysql -uroot \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD}
|
||||
echo "GRANT SELECT, LOCK TABLES, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER ON \\\`${DATABASE_NAME}\\\`.* TO 'root'@'localhost';" | mysql -u\${DATABASE_USERNAME:-root} \${DATABASE_PASSWORD:+-p\$DATABASE_PASSWORD}
|
||||
echo "SET PASSWORD FOR 'root'@'localhost' = PASSWORD('\${DATABASE_PASSWORD:\$DATABASE_PASSWORD}');" | mysql -u root
|
||||
fi
|
||||
fi
|
||||
supervisorctl start foreman >/dev/null
|
||||
|
|
@ -165,9 +175,7 @@ if [ -n "\${DATABASE_INITIAL_CONNECT_MAX_RETRIES}" ]; then
|
|||
fi
|
||||
|
||||
# We may need to try and create a database
|
||||
if [ -n "\${CREATE_DB}" ]; then
|
||||
sudo -u huginn -EH bundle exec rake db:create
|
||||
fi
|
||||
sudo -u huginn -EH bundle exec rake db:create
|
||||
|
||||
# Assuming we have a created database, run the migrations and seed it idempotently.
|
||||
if [ -z "\${DO_NOT_MIGRATE}" ]; then
|
||||
9
docker/multi-process/scripts/standalone-packages
Executable file
9
docker/multi-process/scripts/standalone-packages
Executable file
|
|
@ -0,0 +1,9 @@
|
|||
DEBIAN_FRONTEND=noninteractive
|
||||
apt-get update
|
||||
apt-get install -y python2.7 python-docutils mysql-server \
|
||||
supervisor python-pip && \
|
||||
pip install supervisor-stdout
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
rm -rf /usr/share/doc/
|
||||
rm -rf /usr/share/man/
|
||||
rm -rf /usr/share/locale/
|
||||
|
|
@ -1,38 +0,0 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
locale-gen en_US.UTF-8
|
||||
|
||||
# Shallow clone the huginn project repo
|
||||
git clone --depth 1 https://github.com/cantino/huginn /app
|
||||
|
||||
cd /app
|
||||
|
||||
# add a huginn group and user
|
||||
adduser --group huginn
|
||||
adduser --disabled-login --ingroup huginn --gecos 'Huginn' --no-create-home --home /app huginn
|
||||
adduser huginn sudo
|
||||
passwd -d huginn
|
||||
|
||||
# Change the ownership to huginn
|
||||
chown -R huginn:huginn /app
|
||||
|
||||
# create required tmp and log directories
|
||||
sudo -u huginn -H mkdir -p tmp/pids tmp/cache tmp/sockets log
|
||||
chmod -R u+rwX log tmp
|
||||
|
||||
# install gems required by Huginn, use local cache if available
|
||||
if [ -d "/scripts/cache" ]; then
|
||||
mv /scripts/cache vendor/
|
||||
chown -R huginn:huginn vendor/cache
|
||||
fi
|
||||
sudo -u huginn -H cp .env.example .env
|
||||
sudo -u huginn -H LC_ALL="en_US.UTF-8" LANG="en_US.UTF-8" ON_HEROKU=true bundle install --deployment --without test
|
||||
sudo -u huginn -H rm .env
|
||||
|
||||
# silence setlocale message (THANKS DEBIAN!)
|
||||
cat > /etc/default/locale <<EOF
|
||||
LC_ALL=en_US.UTF-8
|
||||
LANG=en_US.UTF-8
|
||||
EOF
|
||||
|
||||
16
docker/single-process/Dockerfile
Normal file
16
docker/single-process/Dockerfile
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
FROM ubuntu:14.04
|
||||
MAINTAINER Dominik Sander
|
||||
|
||||
ADD scripts/prepare /scripts/prepare
|
||||
RUN /scripts/prepare
|
||||
|
||||
ADD scripts/setup /scripts/setup
|
||||
RUN /scripts/setup
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
ADD scripts/init /scripts/init
|
||||
|
||||
EXPOSE 3000
|
||||
|
||||
CMD ["/scripts/init"]
|
||||
94
docker/single-process/README.md
Normal file
94
docker/single-process/README.md
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
Docker image for Huginn using the production environment and separate container for every process
|
||||
=================================================
|
||||
|
||||
This image runs a linkable [Huginn](https://github.com/cantino/huginn) instance.
|
||||
|
||||
It was inspired by the [official docker container for huginn](https://registry.hub.docker.com/u/cantino/huginn)
|
||||
|
||||
The scripts/init script generates a .env file containing the variables as passed as per normal Huginn documentation.
|
||||
The same environment variables that would be used for Heroku PaaS deployment are used by this script.
|
||||
|
||||
The scripts/init script is aware of mysql and postgres linked containers through the environment variables:
|
||||
|
||||
MYSQL_PORT_3306_TCP_ADDR
|
||||
MYSQL_PORT_3306_TCP_PORT
|
||||
|
||||
and
|
||||
|
||||
POSTGRESQL_PORT_5432_TCP_ADDR
|
||||
POSTGRESQL_PORT_5432_TCP_PORT
|
||||
|
||||
Its recommended to use an image that allows you to create a database via environmental variables at docker run, like `postgresql` or `mysql`, so the db is populated when this script runs.
|
||||
|
||||
Additionally, the database variables may be overridden from the above as per the standard Huginn documentation:
|
||||
|
||||
DATABASE_ADAPTER #(must be either 'postgresql' or 'mysql2')
|
||||
DATABASE_HOST
|
||||
DATABASE_PORT
|
||||
|
||||
This script will run database migrations (rake db:migrate) which should be idempotent.
|
||||
|
||||
It will also seed the database (rake db:seed) unless this is defined:
|
||||
|
||||
DO_NOT_SEED
|
||||
|
||||
This same seeding initially defines the "admin" user with a default password of "password" as per the standard Huginn documentation.
|
||||
|
||||
If you do not wish to have the default 6 agents, you will want to set the above environment variable after your initially deploy, otherwise they will be added automatically the next time a container pointing at the database is spun up.
|
||||
|
||||
The CMD launches Huginn via the scripts/init script. This may become the ENTRYPOINT later. It does take under a minute for Huginn to come up. Use environmental variables that match your DB's creds to ensure it works.
|
||||
|
||||
## Usage
|
||||
|
||||
Simple startup using docker compose (you need to daemonize with `-d` to persist the data):
|
||||
|
||||
cd docker/single-process
|
||||
docker-compose up
|
||||
|
||||
or if you like to use PostgreSQL:
|
||||
|
||||
docker-compose -f postgresql.yml up
|
||||
|
||||
Manual startup and linking to a MySQL container:
|
||||
|
||||
docker run --name huginn_mysql \
|
||||
-e MYSQL_DATABASE=huginn \
|
||||
-e MYSQL_USER=huginn \
|
||||
-e MYSQL_PASSWORD=somethingsecret \
|
||||
-e MYSQL_ROOT_PASSWORD=somethingevenmoresecret \
|
||||
mysql
|
||||
|
||||
docker run --name huginn_web \
|
||||
--link huginn_mysql:mysql \
|
||||
-p 3000:3000 \
|
||||
-e DATABASE_NAME=huginn \
|
||||
-e DATABASE_USERNAME=huginn \
|
||||
-e DATABASE_PASSWORD=somethingsecret \
|
||||
cantino/huginn-single-process
|
||||
|
||||
docker run --name huginn_threaded \
|
||||
--link huginn_mysql:mysql \
|
||||
-e DATABASE_NAME=huginn \
|
||||
-e DATABASE_USERNAME=huginn \
|
||||
-e DATABASE_PASSWORD=somethingsecret \
|
||||
cantino/huginn-single-process /scripts/init bin/threaded.rb
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Other Huginn 12factored environment variables of note, as generated and put into the .env file as per Huginn documentation. All variables of the [.env.example](https://github.com/cantino/huginn/blob/master/.env.example) can be used to override the defaults which a read from the current `.env.example`.
|
||||
|
||||
For variables in the .env.example that are commented out, the default is to not include that variable in the generated .env file.
|
||||
|
||||
## Building on your own
|
||||
|
||||
You don't need to do this on your own, but if you really want:
|
||||
|
||||
docker build --rm=true --tag={yourname}/huginn .
|
||||
|
||||
## Source
|
||||
|
||||
The source is [available on GitHub](https://github.com/cantino/huginn/docker/single-process/).
|
||||
|
||||
Please feel free to submit pull requests and/or fork at your leisure.
|
||||
|
||||
|
||||
35
docker/single-process/develop.yml
Normal file
35
docker/single-process/develop.yml
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
mysqldata:
|
||||
image: mysql
|
||||
command: /bin/true
|
||||
|
||||
mysql:
|
||||
image: mysql
|
||||
volumes_from:
|
||||
- mysqldata
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: myrootpassword
|
||||
MYSQL_DATABASE: huginn
|
||||
MYSQL_USER: huginn
|
||||
MYSQL_PASSWORD: myhuginnpassword
|
||||
|
||||
huginn_web:
|
||||
build: .
|
||||
restart: always
|
||||
extends:
|
||||
file: environment.yml
|
||||
service: huginn_base
|
||||
ports:
|
||||
- 3000:3000
|
||||
links:
|
||||
- mysql
|
||||
|
||||
huginn_threaded:
|
||||
build: .
|
||||
restart: always
|
||||
extends:
|
||||
file: environment.yml
|
||||
service: huginn_base
|
||||
links:
|
||||
- mysql
|
||||
command: /scripts/init bin/threaded.rb
|
||||
|
||||
35
docker/single-process/docker-compose.yml
Normal file
35
docker/single-process/docker-compose.yml
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
mysqldata:
|
||||
image: mysql
|
||||
command: /bin/true
|
||||
|
||||
mysql:
|
||||
image: mysql
|
||||
volumes_from:
|
||||
- mysqldata
|
||||
environment:
|
||||
MYSQL_ROOT_PASSWORD: myrootpassword
|
||||
MYSQL_DATABASE: huginn
|
||||
MYSQL_USER: huginn
|
||||
MYSQL_PASSWORD: myhuginnpassword
|
||||
|
||||
huginn_web:
|
||||
image: cantino/huginn-single-process
|
||||
restart: always
|
||||
extends:
|
||||
file: environment.yml
|
||||
service: huginn_base
|
||||
ports:
|
||||
- 3000:3000
|
||||
links:
|
||||
- mysql
|
||||
|
||||
huginn_threaded:
|
||||
image: cantino/huginn-single-process
|
||||
restart: always
|
||||
extends:
|
||||
file: environment.yml
|
||||
service: huginn_base
|
||||
links:
|
||||
- mysql
|
||||
command: /scripts/init bin/threaded.rb
|
||||
|
||||
7
docker/single-process/environment.yml
Normal file
7
docker/single-process/environment.yml
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
huginn_base:
|
||||
environment:
|
||||
DATABASE_ADAPTER: mysql2
|
||||
DATABASE_NAME: huginn
|
||||
DATABASE_USERNAME: huginn
|
||||
DATABASE_PASSWORD: myhuginnpassword
|
||||
APP_SECRET_TOKEN: 3bd139f9186b31a85336bb89cd1a1337078921134b2f48e022fd09c234d764d3e19b018b2ab789c6e0e04a1ac9e3365116368049660234c2038dc9990513d49c
|
||||
72
docker/single-process/postgresql.yml
Normal file
72
docker/single-process/postgresql.yml
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
postgresdata:
|
||||
image: postgres
|
||||
command: /bin/true
|
||||
|
||||
postgres:
|
||||
image: postgres
|
||||
volumes_from:
|
||||
- postgresdata
|
||||
environment:
|
||||
POSTGRES_PASSWORD: myhuginnpassword
|
||||
POSTGRES_USER: huginn
|
||||
|
||||
huginn_web:
|
||||
image: cantino/huginn-single-process
|
||||
restart: always
|
||||
extends:
|
||||
file: environment.yml
|
||||
service: huginn_base
|
||||
environment:
|
||||
DATABASE_ADAPTER: postgresql
|
||||
ports:
|
||||
- 3000:3000
|
||||
links:
|
||||
- postgres
|
||||
|
||||
huginn_threaded:
|
||||
image: cantino/huginn-single-process
|
||||
restart: always
|
||||
extends:
|
||||
file: environment.yml
|
||||
service: huginn_base
|
||||
environment:
|
||||
DATABASE_ADAPTER: postgresql
|
||||
links:
|
||||
- postgres
|
||||
command: /scripts/init bin/threaded.rb
|
||||
|
||||
# huginn_schedule:
|
||||
# image: cantino/huginn-single-process
|
||||
# extends:
|
||||
# file: environment.yml
|
||||
# service: huginn_base
|
||||
# environment:
|
||||
# DATABASE_ADAPTER: postgresql
|
||||
# links:
|
||||
# - postgres
|
||||
# command: /scripts/init bin/schedule.rb
|
||||
|
||||
|
||||
# huginn_twitter_stream:
|
||||
# image: cantino/huginn-single-process
|
||||
# extends:
|
||||
# file: environment.yml
|
||||
# service: huginn_base
|
||||
# environment:
|
||||
# DATABASE_ADAPTER: postgresql
|
||||
# links:
|
||||
# - postgres
|
||||
# command: /scripts/init bin/twitter_stream.rb
|
||||
|
||||
|
||||
# huginn_dj1:
|
||||
# image: cantino/huginn-single-process
|
||||
# extends:
|
||||
# file: environment.yml
|
||||
# service: huginn_base
|
||||
# environment:
|
||||
# DATABASE_ADAPTER: postgresql
|
||||
# links:
|
||||
# - postgres
|
||||
# command: /scripts/init script/delayed_job run
|
||||
|
||||
57
docker/single-process/scripts/init
Executable file
57
docker/single-process/scripts/init
Executable file
|
|
@ -0,0 +1,57 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
cd /app
|
||||
|
||||
# Configure database based on linked container
|
||||
if [ -n "${MYSQL_PORT_3306_TCP_ADDR}" ]; then
|
||||
DATABASE_ADAPTER=${DATABASE_ADAPTER:-mysql2}
|
||||
DATABASE_HOST=${DATABASE_HOST:-${MYSQL_PORT_3306_TCP_ADDR}}
|
||||
DATABASE_PORT=${DATABASE_PORT:-${MYSQL_PORT_3306_TCP_PORT}}
|
||||
DATABASE_ENCODING=${DATABASE_ENCODING:-utf8mb4}
|
||||
elif [ -n "${POSTGRES_PORT_5432_TCP_ADDR}" ]; then
|
||||
DATABASE_ADAPTER=${DATABASE_ADAPTER:-postgresql}
|
||||
DATABASE_HOST=${DATABASE_HOST:-${POSTGRES_PORT_5432_TCP_ADDR}}
|
||||
DATABASE_PORT=${DATABASE_PORT:-${POSTGRES_PORT_5432_TCP_PORT}}
|
||||
DATABASE_ENCODING=utf8
|
||||
fi
|
||||
|
||||
USE_GRAPHVIZ_DOT=${USE_GRAPHVIZ_DOT:-${USE_GRAPHVIZ_DOT:-dot}}
|
||||
|
||||
# Default to the environment variable values set in .env.example
|
||||
IFS="="
|
||||
grep = /app/.env.example | sed -e 's/^#\([^ ]\)/\1/' | grep -v -e '^#' | \
|
||||
while read var value ; do
|
||||
eval "echo \"$var=\${$var:-\${HUGINN_$var-\$value}}\""
|
||||
done | grep -v -e ^= > /app/.env
|
||||
|
||||
eval "echo PORT=${PORT:-${PORT:-3000}}" >> .env
|
||||
eval "echo RAILS_ENV=${RAILS_ENV:-${RAILS_ENV:-production}}" >> .env
|
||||
eval "echo ON_HEROKU=true" >> .env
|
||||
eval "echo RAILS_SERVE_STATIC_FILES=true" >> .env
|
||||
|
||||
chmod ugo+r /app/.env
|
||||
source /app/.env
|
||||
|
||||
# use default port number if it is still not set
|
||||
case "${DATABASE_ADAPTER}" in
|
||||
mysql2) DATABASE_PORT=${DATABASE_PORT:-3306} ;;
|
||||
postgresql) DATABASE_PORT=${DATABASE_PORT:-5432} ;;
|
||||
*) echo "Unsupported database adapter. Available adapters are mysql2, and postgresql." && exit 1 ;;
|
||||
esac
|
||||
|
||||
sudo -u huginn -H bundle install --without test development --path vendor/bundle
|
||||
|
||||
if [ -z $1 ]; then
|
||||
sudo -u huginn -H bundle exec rake db:create db:migrate RAILS_ENV=${RAILS_ENV}
|
||||
fi
|
||||
|
||||
if [[ -z "${DO_NOT_SEED}" && -z $1 ]]; then
|
||||
sudo -u huginn -H bundle exec rake db:seed RAILS_ENV=${RAILS_ENV}
|
||||
fi
|
||||
|
||||
if [ -z $1 ]; then
|
||||
exec sudo -u huginn -H bundle exec unicorn -c config/unicorn.rb
|
||||
else
|
||||
exec sudo -u huginn -H bundle exec rails runner "$@" RAILS_ENV=${RAILS_ENV}
|
||||
fi
|
||||
40
docker/single-process/scripts/prepare
Executable file
40
docker/single-process/scripts/prepare
Executable file
|
|
@ -0,0 +1,40 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
cat > /etc/dpkg/dpkg.cfg.d/01_nodoc <<EOF
|
||||
# Delete locales
|
||||
path-exclude=/usr/share/locale/*
|
||||
|
||||
# Delete man pages
|
||||
path-exclude=/usr/share/man/*
|
||||
|
||||
# Delete docs
|
||||
path-exclude=/usr/share/doc/*
|
||||
path-include=/usr/share/doc/*/copyright
|
||||
EOF
|
||||
|
||||
export LC_ALL=C
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
minimal_apt_get_install='apt-get install -y --no-install-recommends'
|
||||
|
||||
apt-get update
|
||||
apt-get dist-upgrade -y --no-install-recommends
|
||||
$minimal_apt_get_install software-properties-common
|
||||
add-apt-repository -y ppa:brightbox/ruby-ng
|
||||
apt-get update
|
||||
$minimal_apt_get_install build-essential checkinstall git-core \
|
||||
zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libreadline-dev \
|
||||
libncurses5-dev libffi-dev libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev \
|
||||
graphviz libgraphviz-dev \
|
||||
libmysqlclient-dev libpq-dev libsqlite3-dev \
|
||||
ruby2.2 ruby2.2-dev
|
||||
locale-gen en_US.UTF-8
|
||||
update-locale LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8
|
||||
gem install --no-ri --no-rdoc bundler
|
||||
|
||||
apt-get purge -y python3* rsyslog rsync manpages
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
rm -rf /usr/share/doc/
|
||||
rm -rf /usr/share/man/
|
||||
rm -rf /usr/share/locale/
|
||||
rm -rf /var/log/*
|
||||
36
docker/single-process/scripts/setup
Executable file
36
docker/single-process/scripts/setup
Executable file
|
|
@ -0,0 +1,36 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# add a huginn group and user
|
||||
adduser --group huginn
|
||||
adduser --disabled-login --ingroup huginn --gecos 'Huginn' --no-create-home --home /app huginn
|
||||
passwd -d huginn
|
||||
|
||||
# Shallow clone the huginn project repo
|
||||
git clone --depth 1 https://github.com/cantino/huginn /app
|
||||
|
||||
# Change the ownership to huginn
|
||||
chown -R huginn:huginn /app
|
||||
|
||||
cd app
|
||||
|
||||
# create required tmp and log directories
|
||||
sudo -u huginn -H mkdir -p tmp/pids tmp/cache tmp/sockets log
|
||||
chmod -R u+rwX log tmp
|
||||
|
||||
export LC_ALL=en_US.UTF-8
|
||||
|
||||
# HACK: We need a database connection to precompile the assets, use sqlite for that
|
||||
echo "gem 'sqlite3', '~> 1.3.11'" >> Gemfile
|
||||
sudo -u huginn -H RAILS_ENV=production APP_SECRET_TOKEN=secret DATABASE_ADAPTER=sqlite3 ON_HEROKU=true bundle install --without test development --path vendor/bundle -j 4
|
||||
sudo -u huginn -H RAILS_ENV=production APP_SECRET_TOKEN=secret DATABASE_ADAPTER=sqlite3 ON_HEROKU=true bundle exec rake assets:clean assets:precompile
|
||||
git checkout Gemfile
|
||||
|
||||
# Bundle again to get rid of the sqlite3 gem
|
||||
sudo -u huginn -H ON_HEROKU=true DATABASE_ADAPTER=noop bundle install --without test development --path vendor/bundle
|
||||
|
||||
# Configure the unicorn server
|
||||
mv config/unicorn.rb.example config/unicorn.rb
|
||||
sed -ri 's/^listen .*$/listen ENV["PORT"]/' config/unicorn.rb
|
||||
sed -ri 's/^stderr_path.*$//' config/unicorn.rb
|
||||
sed -ri 's/^stdout_path.*$//' config/unicorn.rb
|
||||
Loading…
Add table
Reference in a new issue