Chapter 10 Launching a Multi-Service Application with Docker Compose, Grafana fails to start

jacqueminv

Hi all,

I'm in chapter 10 Launching a Multi-Service Application with Docker Compose and try to launch the stack defined in prometheus/docker-compose.yml unfortunately I keep getting an error with grafana, it's unable to perform a migration:

logger=settings t=2024-03-28T05:31:29.442497719Z level=info msg="App mode production"
logger=sqlstore t=2024-03-28T05:31:29.44300826Z level=info msg="Connecting to DB" dbtype=sqlite3
logger=migrator t=2024-03-28T05:31:29.443679232Z level=info msg="Starting DB migrations"
logger=migrator t=2024-03-28T05:31:29.444728427Z level=info msg="Executing migration" id="create migration_log table"
logger=migrator t=2024-03-28T05:31:29.448246435Z level=error msg="Executing migration failed" id="create migration_log table" error="database or disk is full" duration=3.511123ms
logger=migrator t=2024-03-28T05:31:29.448347313Z level=error msg="Exec failed" error="database or disk is full" sql="CREATE TABLE IF NOT EXISTS `migration_log` (\n`id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL\n, `migration_id` TEXT NOT NULL\n, `sql` TEXT NOT NULL\n, `success` INTEGER NOT NULL\n, `error` TEXT NOT NULL\n, `timestamp` DATETIME NOT NULL\n);"
Error: ✗ migration failed (id = create migration_log table): no such table: migration_log

Did anyone face similar issue?

My setup:

jacqueminv@sre-lab:~/prometheus$ uname -a
Linux sre-lab 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
jacqueminv@sre-lab:~/prometheus$ sudo docker version
Client: Docker Engine - Community
 Version:           25.0.3
 API version:       1.44
 Go version:        go1.21.6
 Git commit:        4debf41
 Built:             Tue Feb  6 21:14:17 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          25.0.3
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.6
  Git commit:       f417435
  Built:            Tue Feb  6 21:14:17 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

jacqueminv

This enabled me to fix my issue: https://packetpushers.net/blog/ubuntu-extend-your-default-lvm-space/

luisviveropena

Hi @jacqueminv,

I haven't seen that error, and I'm glad you were able to solve it.

Fortunately the error was clear:

error="database or disk is full"

So, it was a bit difficult to fix it as you had LVM, but production systems are known for having it.

Regards,
Luis.