Lab 4.1 Not working. Encrypted Volume is created but I can't attach to instance

AlexandreSousa

Hi I've followed all step for this lab at least 4 times, but when I try to attach the encrypted volume it appears as attaching and then available. No error is shown either in BUI or CLI.

ubuntu@devstack-cc:~/devstack$ openstack volume type list

+--------------------------------------+-------------+-----------+

| ID                                   | Name        | Is Public |

+--------------------------------------+-------------+-----------+

| d5f58df2-a559-4cc0-af7b-540f23a77fd8 | LUKS        | True      |

| 03818bee-9880-44de-892e-92417ffea175 | lvmdriver-1 | True      |

+--------------------------------------+-------------+-----------+

ubuntu@devstack-cc:~/devstack$ cinder encryption-type-list

+--------------------------------------+---------------+-----------------+----------+

------------------+

| Volume Type ID                       | Provider      | Cipher          | Key Size |

 Control Location |

+--------------------------------------+---------------+-----------------+----------+

------------------+

| d5f58df2-a559-4cc0-af7b-540f23a77fd8 | LuksEncryptor | aes-xts-plain64 | 256      |

 front-end        |

+--------------------------------------+---------------+-----------------+----------+

------------------+

ubuntu@devstack-cc:~/devstack$ cinder show crypt-vol

+--------------------------------+--------------------------------------+

| Property                       | Value                                |

+--------------------------------+--------------------------------------+

| attached_servers               | []                                   |

| attachment_ids                 | []                                   |

| availability_zone              | nova                                 |

| bootable                       | false                                |

| consistencygroup_id            | None                                 |

| created_at                     | 2018-01-05T12:32:23.000000           |

| description                    | None                                 |

| encrypted                      | True                                 |

| id                             | d3d53c76-5c90-43b6-86e8-78bd64cf4931 |

| metadata                       |                                      |

| migration_status               | None                                 |

| multiattach                    | False                                |

| name                           | crypt-vol                            |

| os-vol-host-attr:host          | devstack-cc@lvmdriver-1#lvmdriver-1  |

| os-vol-mig-status-attr:migstat | None                                 |

| os-vol-mig-status-attr:name_id | None                                 |

| os-vol-tenant-attr:tenant_id   | 8f776fc3b99f40d1b31e03206f5b7d6a     |

| replication_status             | None                                 |

| size                           | 1                                    |

| snapshot_id                    | None                                 |

| source_volid                   | None                                 |

| status                         | available                            |

| updated_at                     | 2018-01-05T12:36:09.000000           |

| user_id                        | 4cb3d7e3538f4f8dbff60e3f1d074fbf     |

| volume_type                    | LUKS                                 |

+--------------------------------+--------------------------------------+

 

ubuntu@devstack-cc:~/devstack$ openstack volume list

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

| ID                                   | Name      | Status    | Size | Attached to

                   |

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

| d3d53c76-5c90-43b6-86e8-78bd64cf4931 | crypt-vol | available |    1 |

                   |

| ca554042-563d-4949-998f-425a98fd5fa6 |           | in-use    |    1 | Attached to g

olden on /dev/vda  |

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

ubuntu@devstack-cc:~/devstack$ openstack server list

+--------------------------------------+--------+--------+---------------------------

------------------------------+-------+---------+

| f8eba3f9-c95c-45da-bbb3-a37ef3aafc89 | golden | ACTIVE | private=10.0.0.11, fdf9:81

a4:fd37:0:f816:3eff:fef6:a275 |       | m1.tiny |

+--------------------------------------+--------+--------+---------------------------

------------------------------+-------+---------+

ubuntu@devstack-cc:~/devstack$ openstack server add volume f8eba3f9-c95c-45da-bbb3-a3

7ef3aafc89 d3d53c76-5c90-43b6-86e8-78bd64cf4931 --device /dev/vdb

ubuntu@devstack-cc:~/devstack$ openstack volume list

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

| ID                                   | Name      | Status    | Size | Attached to

                   |

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

| d3d53c76-5c90-43b6-86e8-78bd64cf4931 | crypt-vol | available |    1 |

                   |

| ca554042-563d-4949-998f-425a98fd5fa6 |           | in-use    |    1 | Attached to g

olden on /dev/vda  |

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

ubuntu@devstack-cc:~/devstack$ openstack volume list

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

| ID                                   | Name      | Status    | Size | Attached to

                   |

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

| d3d53c76-5c90-43b6-86e8-78bd64cf4931 | crypt-vol | available |    1 |

                   |

| ca554042-563d-4949-998f-425a98fd5fa6 |           | in-use    |    1 | Attached to g

olden on /dev/vda  |

+--------------------------------------+-----------+-----------+------+--------------

-------------------+

ubuntu@devstack-cc:~/devstack$

serewicz

It seems that there were six messages posted then deleted recently. I responded to your first message, but it looks like that is gone.  In case you are wondering why I'm posting this response again, that is why.

What is the output of the logs, such as what you would see with journalctl -a |grep cinder?  Log files are always the first step to troubleshooting, if there is not an obvious error when you run a command.   As the process takes a bit to complete the CLI and BUI would not show the output. 

As you seem to not be seeing my responses I will try to debug this as well and will try to get an answer to you before the message is deleted again.

Regards,

serewicz

I have found the same result, it fails when joining a snapshot instance. If you were to create an instance from an image, a more traditional way, it works. This could be a bug with Cinder or the Nova snapshot process, where it is not providing the necessary drivers and hooks for the volume to be attached.  

Try to create a typical instance and add your encrypted volume to that. If you are able to, then it is indeed a bug.

Regards,

 

serewicz

I have found the same result, it fails when joining a snapshot instance. If you were to create an instance from an image, a more traditional way, it works. This could be a bug with Cinder or the Nova snapshot process, where it is not providing the necessary drivers and hooks for the volume to be attached.  

Try to create a typical instance and add your encrypted volume to that. If you are able to, then it is indeed a bug.

Regards,

 

serewicz

I have found the same result, it fails when joining a snapshot instance. If you were to create an instance from an image, a more traditional way, it works. This could be a bug with Cinder or the Nova snapshot process, where it is not providing the necessary drivers and hooks for the volume to be attached.  

Try to create a typical instance and add your encrypted volume to that. If you are able to, then it is indeed a bug.

Regards,

 

AlexandreSousa

Hi serewicz, thanks for the reply.

I've done as you suggested. I've created an instance fro scratch. Then I've created the crypto volume, but I'm still not being able to attach it to instance.

Regards,

 

 

serewicz

I have run through the process again. If I create an m1.tiny volume all by itself with no snapshot or other outside usage and was able to add a newly created encyrpted volume.   Do you see the same errors and issues when you use a seperate instance and a new encrypted volume?

Regards,

 

AlexandreSousa

I did the same twice. I've launched a new instance from the command line using the cirros image and then I've created a volume type LUKS with encryption following exactly the same steps on course material. Are you using different steps? Could you please post here everything you've done using 100% the CLI? So I could try and reproduce?

PS.: When I use the command openstack server add volume <server> <volume> --device /dev/vdb it comes back to the command prompt (in theory because it worked), but when I run openstack volume list twice the encrypted volume appears initially as "attaching" and one second later "available". 

 

 

serewicz

Hello,

I used the steps from the book. Create a new m1.tiny instance. Create a new, unused, LUKS encrypted volume. Then attach it.   Did you use the encrpted volume that had failed previously?  I have found that once a queue forms to an object in OpenStack following uses fail. Manually deleting everything from the queue and restarting the service should allow following API requests to be honored, but I have found its easier to delete the old volume type and create a new one is faster and there is no need to clear and restart.

Regards, 

AlexandreSousa

Hello,

I used the steps from the book too... I've created a new m1.tiny instance and a new, unused, LUKS encrypted volume. Then when I attach it for a while it appears as "attaching" then after a few seconds appears as "avaliable".

Everytime I do this lab I press the "Start Over" button and only do the encrypted volume lab. (ignoring the other labs).

It is very strange that you follow the same steps from the book and it works and I've tried many times and it does not work.... Last time I've used the pdf and did copy and paste for the whole encryption lab.... so either you are using a different book then I am or is accessing a different lab (one with this fixed).

Could you please copy and paste the steps you've used and the output of each command as I did in the begining of this comment? Do you want to do a webex or use any other remote solution and try to do on the labs provisioned to me?

Thanks

serewicz

Hello,

I also am having issues with consistent joining of the volume. There seems to be an issue with the use of a virtual host and joining the encyprted volume.  I think I was actually breaking an agent for it to show as attached.  But when I try to write data it fails. So my work around was not actually working, it was breaking it instead.  

I will continue to troubleshoot the issue.  It did work, as the output of the book is pasted from the system. As with many things with OpenStack there is constant change. What worked yesterday can be broken today. 

Regards,

AlexandreSousa

Hi, Yeah I agree with your that many things with OpenStack is in a constant change. What worked yesterday can be broken today. 

But when a Customer buys either an Enterprise OpenStack Distro or an Official Training of OpenStack what we expect is stability, consistency, we do not expect something that works today and might broke tomorrow. I mean Linux Foundation need to have a stable repository from where the Openstack is installed for the demo environment. It is not justifiable that a lab depends on this inconsistency.

That is why people become RHCE for example, studying and doing the exam using Red Hat Enteprise Linux in a controlled environment. They do not read something in the book that worked in a certain version of Fedora but not work in a lab with a newer version of Fedora....

Thanks

serewicz

Yes, I agree totally. There are curretnly plans to move from our current setup to a controlled environment instead. I don't know when it will happen, but I hope soon. I will update here when it happens.

Regards,