问题描述
我有一个Timescaledb服务器，在AKS中作为StatefulSet运行.当我删除并重新创建timescaledb pod时，它会出现，即使该pod与最初关联的PV(持久卷)相关联，更改也会丢失.感谢您的帮助.
I have a Timescaledb server running as StatefulSet in AKS. It appears when I delete and recreate timescaledb pod, the changes are lost even though the pod is associated to the initially associated PV (persistent volume). Any help is appreciated.
下面是通过运行kubectl get statefulset timescaledb -o yaml
template:
metadata:
creationTimestamp: null
labels:
app: timescaledb
spec:
containers:
- args:
- -c
- config_file=/etc/postgresql/postgresql.conf
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
key: password
name: timescaledb-secret
image: docker.io/timescale/timescaledb:latest-pg9.6
name: timescaledb-backend
ports:
- containerPort: 5432
name: server
protocol: TCP
resources:
requests:
cpu: "3"
memory: 6Gi
volumeMounts:
- mountPath: /var/lib/postgresql
name: timescaledbdata
- mountPath: /etc/postgresql
name: timescaledb-config
volumes:
- configMap:
defaultMode: 420
name: timescaledb-config
name: timescaledb-config
volumeClaimTemplates:
- metadata:
annotations:
volume.alpha.kubernetes.io/storage-class: standard
creationTimestamp: null
name: timescaledbdata
spec:
accessModes:
- ReadWriteOnce
dataSource: null
resources:
requests:
storage: 200Gi
status:
phase: Pending
下面展示了在吊舱重新创建后创建的临时数据库test_db丢失，并且在整个过程中，吊舱与Azure上的同一PV/磁盘相关联.
Below demonstrates a temp DB test_db created is lost after pod recreation and during the whole process, the pod was associated to the same PV/disk on Azure.
root@e70a91715239:~/keys# k get pvc -l app=timescaledb
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
timescaledbdata-timescaledb-0 Bound pvc-c7eb99cf-6a6b-11e9-b661-be660567cc75 200Gi RWO default 83d
root@e70a91715239:~/keys# k exec -ti timescaledb-0 bash
bash-4.4# psql -U postgres;
psql (9.6.13)
Type "help" for help.
postgres=# create database test_db;
CREATE DATABASE
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
test_db | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
(4 rows)
root@e70a91715239:~/keys# k get pods | grep timescale
timescaledb-0 1/1 Running 0 12m
root@e70a91715239:~/keys# k delete pod/timescaledb-0
pod "timescaledb-0" deleted
root@e70a91715239:~/keys# k get pods | grep timescale
timescaledb-0 1/1 Running 0 14s
root@e70a91715239:~/keys# k exec -ti timescaledb-0 bash
bash-4.4# psql -U postgres
psql (9.6.13)
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
root@e70a91715239:~/keys# k get pvc -l app=timescaledb
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
timescaledbdata-timescaledb-0 Bound pvc-c7eb99cf-6a6b-11e9-b661-be660567cc75 200Gi RWO default 83d
可能正在按提示重新初始化.请参阅日志.有关为什么这样做的任何指示.
Possibly it is reinitialising as hinted. Please see logs. Any pointers on why it would do that.
更新1:我看了看timescale窗格中的挂载，它似乎对/var/lib/postgresql和/var/lib/postgresql/data具有不同的分区.我不明白为什么.
Update 1:I had a look at the mounts in the timescale pod, it seems to have different partitions for /var/lib/postgresql and /var/lib/postgresql/data. I do not understand why.
Filesystem Size Used Available Use% Mounted on
overlay 96.9G 22.1G 74.8G 23% /
tmpfs 64.0M 0 64.0M 0% /dev
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/sda1 96.9G 22.1G 74.8G 23% /docker-entrypoint-initdb.d
/dev/sda1 96.9G 22.1G 74.8G 23% /dev/termination-log
shm 64.0M 4.0K 64.0M 0% /dev/shm
/dev/sda1 96.9G 22.1G 74.8G 23% /etc/resolv.conf
/dev/sda1 96.9G 22.1G 74.8G 23% /etc/hostname
/dev/sda1 96.9G 22.1G 74.8G 23% /etc/hosts
/dev/sdc 196.7G 59.3M 196.7G 0% /var/lib/postgresql
/dev/sda1 96.9G 22.1G 74.8G 23% /var/lib/postgresql/data
不了解在以下配置中上述挂载如何发生
Do not understand how the above mounts happen for below config
volumeMounts:
- mountPath: /var/lib/postgresql
name: timescaledbdata
- mountPath: /etc/postgresql
name: timescaledb-config
推荐答案
问题是在postgres:9.6 Dockerfile中，针对/var/lib/postgresql/data的VOLUME声明导致容器上的附加挂载.当我们在/var/lib/postgresql处有卷挂载时，该挂载是短暂的.但是我们无法将AKS卷挂载到/var/lib/postgresql/data，因为该卷带有lost+found子目录，而Postgres希望使用空目录来存储DB文件.
The problem was there was a VOLUME declaration for /var/lib/postgresql/data in postgres:9.6 Dockerfile which caused additional mount on the container. That mount was ephemeral when we had the volume mount at /var/lib/postgresql. But we were not able to mount AKS volume to /var/lib/postgresql/data because the volume came with lost+found subdirectory and Postgres expects empty directory to store DB files.
解决方法是将卷安装在/var/lib/postgresql/data上，并告诉Postgres使用/var/lib/postgresql/data下的子目录来存储具有PGDATA env var的文件.
The fix was to mount volume at /var/lib/postgresql/data and tell Postgres to use a subdirectory underneath /var/lib/postgresql/data to store files with PGDATA env var.
以下是k8s statefulset配置中修复的相关部分
Below are the relevant parts of fix in the k8s statefulset configuration
env:
- name: PGDATA
value: "/var/lib/postgresql/data/dbfiles"
...
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: timescaledata
这篇关于kubernetes timescaledb statefulset:吊舱重新创建丢失的更改的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！