2017-12-15 version Lab 3.2 problem

jonathanemett

I have retried the labs since the new version based on pike has come out.  I have got though to Lab 3.2 step 5, with no errors seen in the ./stack.sh output (and also in the stack.sh.log file):

=========================

DevStack Component Timing

=========================

Total runtime    781

run_process        6

apt-get-update     9

pip_install      330

osc                2

wait_for_service   8

git_timed        132

apt-get          176

=========================

This is your host IP address: 192.168.97.2

This is your host IPv6 address: ::1

WARNING:

Using lib/neutron-legacy is deprecated, and it will be removed in the future

Services are running under systemd unit files.

For more information see:

https://docs.openstack.org/devstack/latest/systemd.html

DevStack Version: pike

Change: 4a85d5d6e040b08e085f9f5752de90a2346df6d1 Move remove_uwsgi_config to cleanup_

placement 2017-12-06 11:31:12 +0000

OS Version: Ubuntu 16.04 xenial

 

However, when I try and run the nova-manage command as I only see one hypervisor listed, I get the following output:

ubuntu@compute-node:~/devstack$ source openrc admin

WARNING: setting legacy OS_TENANT_NAME to support cli tools.

ubuntu@compute-node:~/devstack$ openstack hypervisor list

+----+---------------------+-----------------+--------------+-------+

| ID | Hypervisor Hostname | Hypervisor Type | Host IP      | State |

+----+---------------------+-----------------+--------------+-------+

|  1 | devstack-cc         | QEMU            | 192.168.97.1 | up    |

+----+---------------------+-----------------+--------------+-------+

ubuntu@compute-node:~/devstack$ nova-manage cell_v2 discover_hosts

An error has occurred:

Traceback (most recent call last):

  File "/opt/stack/nova/nova/cmd/manage.py", line 1797, in main

    ret = fn(*fn_args, **fn_kwargs)

  File "/opt/stack/nova/nova/cmd/manage.py", line 1583, in discover_hosts

    hosts = host_mapping_obj.discover_hosts(ctxt, cell_uuid, status_fn)

  File "/opt/stack/nova/nova/objects/host_mapping.py", line 206, in discover_hosts

    cell_mappings = objects.CellMappingList.get_all(ctxt)

  File "/usr/local/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 1

84, in wrapper

    result = fn(cls, context, *args, **kwargs)

  File "/opt/stack/nova/nova/objects/cell_mapping.py", line 137, in get_all

    db_mappings = cls._get_all_from_db(context)

  File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", l

ine 978, in wrapper

    with self._transaction_scope(context):

  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__

    return self.gen.next()

  File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", l

ine 1028, in _transaction_scope

    context=context) as resource:

  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__

    return self.gen.next()

  File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", l

ine 633, in _session

    bind=self.connection, mode=self.mode)

  File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", l

ine 398, in _create_session

    self._start()

  File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", l

ine 484, in _start

    engine_args, maker_args)

  File "/usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", l

ine 506, in _setup_for_connection

    "No sql_connection parameter is established")

CantStartEngineError: No sql_connection parameter is established

I've checked the local.conf and done several unstack/clean cycles, but am stuck here.  What investigation I've managed to do implies the compute-node configuration isn't being added to a cell(?), but this looks to be a bug.

Does ANYONE have any ideas on this.

serewicz

Hello,

I just ran the labs and it seems a script is missing important variable. This is the local.conf file on the compute-node 

[[local|localrc]]
HOST_IP=192.168.97.2 # IP for compute-node
SERVICE_HOST=192.168.97.1 # devstack-cc IP, first node you used
FLAT_INTERFACE=eth0
FIXED_RANGE=10.4.128.0/20
FIXED_NETWORK_SIZE=4096
FLOATING_RANGE=192.168.42.128/25
MULTI_HOST=1
LOGFILE=/opt/stack/logs/stack.sh.log
ADMIN_PASSWORD=openstack
DATABASE_PASSWORD=db-secret
RABBIT_PASSWORD=rb-secret
SERVICE_PASSWORD=sr-secret
DATABASE_TYPE=mysql
MYSQL_HOST=
RABBIT_HOST=
GLANCE_HOSTPORT=:9292
ENABLED_SERVICES=n-cpu,q-agt,n-api-meta,c-vol,placement-client
NOVA_VNC_ENABLED=True
NOVNCPROXY_URL="http://:6080/vnc_auto.html"
VNCSERVER_LISTEN=
VNCSERVER_PROXYCLIENT_ADDRESS=

You will note that MYSQL_HOST, and a few others is empty. When the nova-manage command runs it tries to connect to the database to update the hypervisor entry.  The local.conf file should have several entries of the variable $SERVICE_HOST.  If you reference the lab Exercise 3.2, Install Software on the New Compute Node, step 2 you will see what the local.conf file should look like.

I will report this to the infrastructure folks, to update the script. ./unstack.sh, ./clean.sh, edit the local.conf file and ./stack should fix this issue.  I am about to try it right now, just wanted to get you an answer ASAP.

Regards,

serewicz

Hello again,

In addition to the script issue, which is rolling out now, there has been a change from the DevStack folks on how to add another node to the system. The nova-manage cell_v2 discover_hosts process was a work-around for a previous bug. The new five step process, which I am putting into the course book now, is as follows:

When the *compute-node* finishes the ./stack.sh then on the **devstack-cc** run this process:

1) Source the config file as admin

    ubuntu@devstack-cc:~$ cd devstack/

    ubuntu@devstack-cc:~/devstack$ source openrc admin
    WARNING: setting legacy OS_TENANT_NAME to support cli tools.


2) Verify the compute-host was added and is up.

    ubuntu@devstack-cc:~/devstack$ nova service-list --binary nova-compute
    +--------------------------------------+--------------+--------------+------+--------
    -+-------+----------------------------+-----------------+-------------+
    | Id                                   | Binary       | Host         | Zone | Status
     | State | Updated_at                 | Disabled Reason | Forced down |
    +--------------------------------------+--------------+--------------+------+--------
    -+-------+----------------------------+-----------------+-------------+
    | 32fa0ccd-45a8-45f5-b2e7-2d84e7377eb3 | nova-compute | devstack-cc  | nova | enabled
     | up    | 2017-12-19T19:49:35.000000 | -               | False       |
    | 9fc15015-d150-478f-ba4f-764fe4ed03c9 | nova-compute | compute-node | nova | enabled
     | up    | 2017-12-19T19:49:35.000000 | -               | False       |
    +--------------------------------------+--------------+--------------+------+--------
    -+-------+----------------------------+-----------------+-------------+


3) Verify the hypervisor has not yet been added.

    ubuntu@devstack-cc:~/devstack$ openstack hypervisor list
    +----+---------------------+-----------------+--------------+-------+
    | ID | Hypervisor Hostname | Hypervisor Type | Host IP      | State |
    +----+---------------------+-----------------+--------------+-------+
    |  1 | devstack-cc         | QEMU            | 192.168.97.1 | up    |
    +----+---------------------+-----------------+--------------+-------+


4) Use a script to join the hypervisor to the cloud.

    ubuntu@devstack-cc:~/devstack$ ./tools/discover_hosts.sh
    /usr/local/lib/python2.7/dist-packages/pymysql/cursors.py:166: Warning: (1287, u"'@@t
    x_isolation' is deprecated and will be removed in a future release. Please use '@@tra
    nsaction_isolation' instead")
      result = self._query(query)
    Found 2 cell mappings.
    Skipping cell0 since it does not contain hosts.
    Getting compute nodes from cell 'cell1': 79bd3053-a007-469d-ba72-d7b106d08568
    Found 1 unmapped computes in cell: 79bd3053-a007-469d-ba72-d7b106d08568
    Checking host mapping for compute host 'compute-node': b3caa6f3-fe33-49af-839a-375813
    8af2b1
    Creating host mapping for compute host 'compute-node': b3caa6f3-fe33-49af-839a-375813
    8af2b1


5) Verify the compute-host has been added.

    ubuntu@devstack-cc:~/devstack$ openstack hypervisor list
    +----+---------------------+-----------------+--------------+-------+
    | ID | Hypervisor Hostname | Hypervisor Type | Host IP      | State |
    +----+---------------------+-----------------+--------------+-------+
    |  1 | devstack-cc         | QEMU            | 192.168.97.1 | up    |
    |  2 | compute-node        | QEMU            | 192.168.97.2 | up    |
    +----+---------------------+-----------------+--------------+-------+
    ubuntu@devstack-cc:~/devstack$

I have just tested this process and it worked upon a fresh load of lab 3.2

Regards,

jonathanemett

OK, I have followed these instructions and confirm that they work.  Thanks for the quick response.

I have just looked at the full Labs and Solutions doc (from 1.8) and the Specific Lab and Solution doc (from 3.18.b), and they as of now do not have the updated multi-node instructions.

Also RE: the local.conf.  I was copy & pasting the example in full Labs and Solutions which I downloaded yesterday (19 Dec), so it DID in fact have the $SERVICE_HOST details included.

coop

Hi:

It takes some time to update the course as it cannot be done in real time (even a small change requires an entire upload) Forums help and I'm sure there will be a new errata file posted shortly  It should not be long before there is a new update to the course itself, but we have to let things accumulate for a while.

serewicz

Yes, the local.conf file in the book was accurate. Due to an issue with the script used to install the lab node the local.conf on the node was inaccurate. As I was troubleshooting the issue I found the book and the node did not match. Should you have restacked without editing the local.conf file on the node it would have continued to fail. If you edited the local.conf with the file from the book then it would have installed correctly, but then would have encountered the five-step update. 

Regards,