跳转至

V2.3.0升级特性opentenbase_subscription使用说明

1. 修改wal_level

因为多活使用的是逻辑复制来实现,因此需要配置多活的所有集群中,所有CN/DN节点的 wal_level 全部设置为 logical

vim postgres.comf
wal_level = logical
:wq

2. 安装 opentenbase_subscription 工具:

连接配置多活的2个集群分别各自的CN(针对任意集群,选择任意一个CN执行即可,但是2个集群都需要安装)执行:

CREATE EXTENSION opentenbase_subscription;

3. 单向同步配置

这里以发布端1CN+2DN,订阅端1CN+2DN为例,其他集群规模配置步骤相似

3.1 连接发布端任意CN建表

postgres=# create table users(id int,name varchar(30),f3 int,PRIMARY KEY(id));
postgres=# insert into users select i, 'a', i+1 from generate_series(1, 399999) i;
INSERT 0 399999

3.2 在发布端 DN上进行数据发布

注意:如果有发布端集群有多个DN,则每个DN都需要进行配置发布

发布端DN1:

CREATE PUBLICATION mypub_dn001 FOR ALL TABLES;

发布端DN2:

CREATE PUBLICATION mypub_dn002 FOR ALL TABLES;

3.3 在订阅端 CN上进行数据订阅

订阅端CN:

create table users(id int,name varchar(30),f3 int,PRIMARY KEY(id)) ;

CREATE OPENTENBASE SUBSCRIPTION sub_dn001 CONNECTION 'host=100.105.39.157 port=20008 user=jennyer dbname=postgres' PUBLICATION mypub_dn001 WITH (connect=true, enabled=true, create_slot=true, copy_data=true, synchronous_commit=on, ignore_pk_conflict = true, parallel_number=4);

CREATE OPENTENBASE SUBSCRIPTION sub_dn002 CONNECTION 'host=100.105.39.157 port=20009 user=jennyer dbname=postgres' PUBLICATION mypub_dn002 WITH (connect=true, enabled=true, create_slot=true, copy_data=true, synchronous_commit=on, ignore_pk_conflict = true, parallel_number=4);

查询验证从发布端同步过来的存量数据

订阅端CN:

select count(*) from users;
 count  
--------
 399999
(1 row)

3.4 验证增量数据同步

连接发布端CN插入增量数据:

insert into users select i, 'a', i+1 from generate_series(400000, 500000) i;
INSERT 0 100001

查询验证从发布端同步过来的增量数据
订阅端CN:

select count(*) from users;
 count  
--------
 500000
(1 row)

4. 双向同步配置

4.1 双活集群各自建表

集群1-CN:

create table test(id int,name varchar(30),PRIMARY KEY(id));

集群2-CN:

create table test(id int,name varchar(30),PRIMARY KEY(id));

4.2 集群1最为发布端,在所有DN上进行数据发布

集群1-DN1:

CREATE PUBLICATION cluster1_dn001 FOR ALL TABLES;

集群1-DN2:

CREATE PUBLICATION cluster1_dn002 FOR ALL TABLES;

4.3 集群2最为发布端,在所有DN上进行数据发布

集群2-DN1:

CREATE PUBLICATION cluster2_dn001 FOR ALL TABLES;

集群2-DN2:

CREATE PUBLICATION cluster2_dn002 FOR ALL TABLES;

4.4 集群1最为订阅端, 在任意CN上进行数据订阅

集群1-CN:

CREATE OPENTENBASE SUBSCRIPTION sub_cluster2_dn001 CONNECTION 'host=100.105.50.198 port=20008 user=jennyer dbname=postgres' PUBLICATION cluster2_dn001 WITH (connect=true, enabled=true, create_slot=true, copy_data=true, synchronous_commit=on, ignore_pk_conflict = true, parallel_number=4);
CREATE OPENTENBASE SUBSCRIPTION sub_cluster2_dn002 CONNECTION 'host=100.105.50.198 port=20009 user=jennyer dbname=postgres' PUBLICATION cluster2_dn002 WITH (connect=true, enabled=true, create_slot=true, copy_data=true, synchronous_commit=on, ignore_pk_conflict = true, parallel_number=4);

4.5 集群2最为订阅端, 在任意CN上进行数据订阅

集群2-CN:

CREATE OPENTENBASE SUBSCRIPTION sub_cluster1_dn001 CONNECTION 'host=100.105.39.157 port=20008 user=jennyer dbname=postgres' PUBLICATION cluster1_dn001 WITH (connect=true, enabled=true, create_slot=true, copy_data=true, synchronous_commit=on, ignore_pk_conflict = true, parallel_number=4);
CREATE OPENTENBASE SUBSCRIPTION sub_cluster1_dn002 CONNECTION 'host=100.105.39.157 port=20009 user=jennyer dbname=postgres' PUBLICATION cluster1_dn002 WITH (connect=true, enabled=true, create_slot=true, copy_data=true, synchronous_commit=on, ignore_pk_conflict = true, parallel_number=4);

4.6 集群1中进行写入部分数据

集群1-CN:

insert into test values(1,'a'),(3,'a'),(5,'a'),(7,'a'),(9,'a'),(11,'a'),(13,'a'),(15,'a');

4.7 集群2中进行写入部分数据

集群2-CN:

insert into test values(2,'b'),(4,'b'),(6,'b'),(8,'b'),(10,'b'),(12,'b'),(14,'b'),(16,'b');

4.8 验证集群1同步的结果(数据来源本端写入和远端同步)

集群1-CN:

select  * from test;
postgres=# select  * from test order by 1;
 id | name 
----+------
  1 | a
  2 | b
  3 | a
  4 | b
  5 | a
  6 | b
  7 | a
  8 | b
  9 | a
 10 | b
 11 | a
 12 | b
 13 | a
 14 | b
 15 | a
 16 | b
(16 rows)

4.9 验证集群2同步的全量结果(数据来源本端写入和远端同步)

集群2-CN:

postgres=# select  * from test order by 1;
 id | name 
----+------
  1 | a
  2 | b
  3 | a
  4 | b
  5 | a
  6 | b
  7 | a
  8 | b
  9 | a
 10 | b
 11 | a
 12 | b
 13 | a
 14 | b
 15 | a
 16 | b
(16 rows)

可以在集群1或者2上任意一方进行写入,最终集群1和2都有着全量数据。