本文共 13400 字，大约阅读时间需要 44 分钟。
关于什么是mongodb分片集群，这些这里就不介绍，网上有很多的说明。
由于是测试资源有限，于是就把所有的节点都搭建在一台服务器上面了，实际的生产环境最好config/mongos/shard都分开安装到不同的机器上，这里搭建了12个节点的分片集群；将端口区分开，创建3个mongs节点，3节点的config-server副本集，3个2节点的shard副本集。使用版本：4.0.18
| 主机 | 端口 | 角色 | 副本集/分片类型 |
| 172.100.1.35 | 27017,37017,47017 | mongos | — |
| 172.100.1.35 | 27019,37019,47019 | config server | cs0/configsvr |
| 172.100.1.35 | 27018,37018,47018 | shardA | shardA/shardsvr |
| 172.100.1.35 | 27016,37016,47016 | shardB | shardB/shardsvr |
话不多说直接开搞:
下面就是此次测试环境需要的mongodb节点
[root@cpe-172-100-1-35 jsunicom]# lltotal 0drwxr-xr-x 6 root root 203 Jun 2 16:10 mongodb_config_27019drwxr-xr-x 6 root root 203 Jun 2 11:28 mongodb_config_37019drwxr-xr-x 6 root root 203 Jun 2 11:28 mongodb_config_47019drwxr-xr-x 6 root root 198 Jun 2 14:31 mongodb_mongos1drwxr-xr-x 6 root root 198 Jun 2 14:31 mongodb_mongos2drwxr-xr-x 6 root root 198 Jun 2 14:30 mongodb_mongos3drwxr-xr-x 6 root root 199 Jun 2 14:38 mongodb_shardA_1drwxr-xr-x 6 root root 199 Jun 2 14:38 mongodb_shardA_2drwxr-xr-x 6 root root 199 Jun 2 14:02 mongodb_shardA_3drwxr-xr-x 6 root root 199 Jun 2 14:22 mongodb_shardB_1drwxr-xr-x 6 root root 199 Jun 2 14:22 mongodb_shardB_2drwxr-xr-x 6 root root 199 Jun 2 14:22 mongodb_shardB_3
首先对configsvr进行配置
分别在 mongodb_config_27019，mongodb_config_37019，mongodb_config_47019下创建对应的配置文件，这里我的配置文件分别为:mogondb_config_27019.conf，mogondb_config_37019.conf，mogondb_config_47019.conf
编辑配置文件，这里就以添加mongodb_config_27019几点为例，添加以下内容:
[root@cpe-172-100-1-35 mongodb_config_27019]# cat mogondb_config_27019.confsystemLog: traceAllExceptions: false path: /webapp/jsunicom/mongodb_config_27019/logs/mongod.log logAppend: true destination: file timeStampFormat: ctimeprocessManagement: fork: true pidFilePath: /webapp/jsunicom/mongodb_config_27019/tmp/mongod.pidnet: port: 27019 bindIp: 172.100.1.35 maxIncomingConnections: 20000storage: dbPath: /webapp/jsunicom/mongodb_config_27019/data directoryPerDB: true wiredTiger: engineConfig: cacheSizeGB: 1 directoryForIndexes: trueoperationProfiling: slowOpThresholdMs: 1000replication: oplogSizeMB: 2048 replSetName: cs0sharding: clusterRole: configsvr
注意：其余2个节点除了端口号以及路径和上面的不一样，其余完全一致。
配置好之后，分别启动这3个config节点
[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_config_27019/bin/mongod -f /webapp/jsunicom/mongodb_config_27019/mogondb_config_27019.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_config_37019/bin/mongod -f /webapp/jsunicom/mongodb_config_37019/mogondb_config_37019.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_config_47019/bin/mongod -f /webapp/jsunicom/mongodb_config_47019/mogondb_config_47019.conf
启动之后，随便登录上面3个config节点中的一个进行初始化
[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_config_27019/bin/mongo --host 172.100.1.35 --port 27019执行以下命令use adminrs.initiate( { _id: "cs0", configsvr: true, members: [ { _id : 0, host : "172.100.1.35:27019" }, { _id : 1, host : "172.100.1.35:37019" }, { _id : 2, host : "172.100.1.35:47019" } ] })执行成功之后，一定要用以下命令来查看初始化是否正常rs.status()其中health：1 代表初始化成功 到此config配置结束
下面对shardA,shardB配置：
shardA配置:
分别在mongodb_shardA_1，mongodb_shardA_2，mongodb_shardA_3下创建对应的配置文件
我这里分别为：mongodb_shardA_1.conf，mongodb_shardA_2.conf ，mongodb_shardA_3.conf
编辑配置文件，这里还是以mongodb_shardA_1.conf为例，添加如下内容：
[root@cpe-172-100-1-35 mongodb_shardA_1]# cat mongodb_shardA_1.conf systemLog: traceAllExceptions: false path: /webapp/jsunicom/mongodb_shardA_1/logs/mongod.log logAppend: true destination: file timeStampFormat: ctimeprocessManagement: fork: true pidFilePath: /webapp/jsunicom/mongodb_shardA_1/tmp/mongod.pidnet: port: 27018 bindIp: 172.100.1.35 maxIncomingConnections: 20000storage: dbPath: /webapp/jsunicom/mongodb_shardA_1/data directoryPerDB: true wiredTiger: engineConfig: cacheSizeGB: 1 directoryForIndexes: trueoperationProfiling: slowOpThresholdMs: 1000replication: oplogSizeMB: 3072 replSetName: shardAsharding: clusterRole: shardsvrsetParameter: connPoolMaxShardedInUseConnsPerHost: 100 shardedConnPoolIdleTimeoutMinutes: 10 connPoolMaxInUseConnsPerHost: 100 globalConnPoolIdleTimeoutMinutes: 10 maxIndexBuildMemoryUsageMegabytes: 2048
备注：其余2个节点配置除了端口号以及路径不一致以外，其余完全和上面一致
shardB配置:
其实shardB配置和shardA完全一样，只要注意端口以及路径就行了
同样分别在mongodb_shardB_1，mongodb_shardB_2，mongodb_shardB_3下创建对应的配置文件
我这里分别为：mongodb_shardB_1.conf，mongodb_shardB_2.conf ，mongodb_shardB_3.conf
编辑配置文件，这里还是以mongodb_shardB_1.conf为例，添加如下内容：
[root@cpe-172-100-1-35 mongodb_shardB_1]# cat mongodb_shardB_1.conf systemLog: traceAllExceptions: false path: /webapp/jsunicom/mongodb_shardB_1/logs/mongod.log logAppend: true destination: file timeStampFormat: ctimeprocessManagement: fork: true pidFilePath: /webapp/jsunicom/mongodb_shardB_1/tmp/mongod.pidnet: port: 27016 bindIp: 172.100.1.35 maxIncomingConnections: 20000storage: dbPath: /webapp/jsunicom/mongodb_shardB_1/data directoryPerDB: true wiredTiger: engineConfig: cacheSizeGB: 1 directoryForIndexes: trueoperationProfiling: slowOpThresholdMs: 1000replication: oplogSizeMB: 3072 replSetName: shardBsharding: clusterRole: shardsvrsetParameter: connPoolMaxShardedInUseConnsPerHost: 100 shardedConnPoolIdleTimeoutMinutes: 10 connPoolMaxInUseConnsPerHost: 100 globalConnPoolIdleTimeoutMinutes: 10 maxIndexBuildMemoryUsageMegabytes: 2048
备注：其余2个节点配置除了端口号以及路径不一致以外，其余完全和上面一致
配置好之后，那就把上面的shardA3个节点以及shardB三个节点分别起起来。
[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_shardA_1/bin/mongod -f /webapp/jsunicom/mongodb_shardA_1/mongodb_shardA_1.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_shardA_2/bin/mongod -f /webapp/jsunicom/mongodb_shardA_2/mongodb_shardA_2.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_shardA_3/bin/mongod -f /webapp/jsunicom/mongodb_shardA_3/mongodb_shardA_3.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_shardB_1/bin/mongod -f /webapp/jsunicom/mongodb_shardB_1/mongodb_shardB_1.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_shardB_2/bin/mongod -f /webapp/jsunicom/mongodb_shardB_2/mongodb_shardB_2.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_shardB_3/bin/mongod -f /webapp/jsunicom/mongodb_shardB_3/mongodb_shardB_3.conf
然后分别初始化话replicat 配置
shardA:
随便登录shardA3个节点中的其中一个节点即可：
[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_config_27019/bin/mongo --host 172.100.1.35 --port 27018登录之后执行以下命令use adminrs.initiate( { _id : "shardA", members: [ { _id : 0, host : "172.100.1.35:27018" }, { _id : 1, host : "172.100.1.35:37018" }, { _id : 2, host : "172.100.1.35:47018",arbiterOnly: true } ] })同样也要用下面命令去查看初始化是否成功rs.status()其中health：1 代表初始化成功 shardB:
随便登录shardB3个节点中的其中一个节点即可：
[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_config_27019/bin/mongo --host 172.100.1.35 --port 27016登录成功以后执行以下命令进行初始化use adminrs.initiate( { _id : "shardB", members: [ { _id : 0, host : "172.100.1.35:27016" }, { _id : 1, host : "172.100.1.35:37016" }, { _id : 2, host : "172.100.1.35:47016",arbiterOnly: true } ] })同样需要使用rs.status()命令查看是否执行成功其中health ：1 代表初始化成功 至此shardA，shardB分片配置到此结束
最后配置mongos
这里mongos也做了3个节点，
分别在mongodb_mongos1，mongodb_mongos2，mongodb_mongos3下配置mongodb_mongos1.conf，mongodb_mongos2.conf，mongodb_mongos3.conf
编辑配置文件，这里同样以mongodb_mongos1.conf为例，添加如下内容:
[root@cpe-172-100-1-35 mongodb_mongos1]# cat mongodb_mongos1.conf systemLog: traceAllExceptions: false path: /webapp/jsunicom/mongodb_mongos1/logs/mongos.log logAppend: true destination: file timeStampFormat: ctimeprocessManagement: fork: true pidFilePath: /webapp/jsunicom/mongodb_mongos1/tmp/mongos.pidnet: port: 27017 bindIp: 172.100.1.35 maxIncomingConnections: 20000operationProfiling: slowOpThresholdMs: 1000replication: localPingThresholdMs: 300sharding: configDB: cs0/172.100.1.35:27019,172.100.1.35:37019,172.100.1.35:47019setParameter: connPoolMaxShardedInUseConnsPerHost: 100 shardedConnPoolIdleTimeoutMinutes: 10 connPoolMaxInUseConnsPerHost: 100 globalConnPoolIdleTimeoutMinutes: 10
备注：其余2个节点配置除了端口号以及路径和上面不一样之外，其余完全一致
上面配置文件配置好之后，分别启动这3个节点：
[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_mongos1/bin/mongos -f /webapp/jsunicom/mongodb_mongos1/mongodb_mongos1.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_mongos2/bin/mongos -f /webapp/jsunicom/mongodb_mongos2/mongodb_mongos2.conf[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_mongos3/bin/mongos -f /webapp/jsunicom/mongodb_mongos3/mongodb_mongos3.conf
随便登录上面3个节点中的其中一个，添加replicat 到sharding中
[root@cpe-172-100-1-35 ~]# /webapp/jsunicom/mongodb_config_27019/bin/mongo --host 172.100.1.35 --port 27017登录之后执行如下命令:use adminsh.addShard("shardA/172.100.1.35:27018,172.100.1.35:37018,172.100.1.35:47018")sh.addShard("shardA/172.100.1.35:27018,172.100.1.35:37016,172.100.1.35:47016")执行完之后使用rs.status()命令查看状态mongos> sh.status()--- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5ed5e536856e004008317884") } shards: { "_id" : "shardA", "host" : "shardA/172.100.1.35:27018,172.100.1.35:37018", "state" : 1 } { "_id" : "shardB", "host" : "shardB/172.100.1.35:27016,172.100.1.35:37016", "state" : 1 } active mongoses: "4.0.18" : 3 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shardB 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shardB Timestamp(1, 0) 到此所有的配置都结束了，下面来进行测试以下:
下面所有的操作都是在mongos其中一个节点上执行的
启动sharding库:
sh.enableSharding("yuhuashi")创建分片集合：
sh.shardCollection("yuhuashi.user",{"name" : 1}) #ranged 分片 sh.shardCollection("yuhuashi.user", { "task_id" : "hashed" } ) #hash分片我这里使用了hashed分片
mongos> sh.shardCollection("yuhuashi.user", { "task_id" : "hashed" } ){ "collectionsharded" : "yuhuashi.user", "collectionUUID" : UUID("43d3cce7-0225-4dcc-8b7e-aecc7b3932b6"), "ok" : 1, "operationTime" : Timestamp(1591081873, 27), "$clusterTime" : { "clusterTime" : Timestamp(1591081873, 27), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }} 可以使用sh.status()来查看状态
mongos> sh.status()--- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5ed5e536856e004008317884") } shards: { "_id" : "shardA", "host" : "shardA/172.100.1.35:27018,172.100.1.35:37018", "state" : 1 } { "_id" : "shardB", "host" : "shardB/172.100.1.35:27016,172.100.1.35:37016", "state" : 1 } active mongoses: "4.0.18" : 3 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: No recent migrations databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shardB 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shardB Timestamp(1, 0) { "_id" : "yuhuashi", "primary" : "shardA", "partitioned" : true, "version" : { "uuid" : UUID("fb60172b-49e9-4d3a-8052-1e8780b0ff52"), "lastMod" : 1 } } yuhuashi.user shard key: { "task_id" : "hashed" } unique: false balancing: true chunks: shardA 2 shardB 2 { "task_id" : { "$minKey" : 1 } } -->> { "task_id" : NumberLong("-4611686018427387902") } on : shardA Timestamp(1, 0) { "task_id" : NumberLong("-4611686018427387902") } -->> { "task_id" : NumberLong(0) } on : shardA Timestamp(1, 1) { "task_id" : NumberLong(0) } -->> { "task_id" : NumberLong("4611686018427387902") } on : shardB Timestamp(1, 2) { "task_id" : NumberLong("4611686018427387902") } -->> { "task_id" : { "$maxKey" : 1 } } on : shardB Timestamp(1, 3) 下面来测试插入测试数据看以下
mongos> use yuhuashiswitched to db yuhuashimongos> show tables;usermongos> db.user.find().count()0mongos> for(i=1;i<=10000;i++){db.user.insert({"task_id":i,"name":"shiyu"+i,"age":i})}WriteResult({ "nInserted" : 1 })
分别登录shardA和shardB分片查看
shardA:PRIMARY> use yuhuashiswitched to db yuhuashishardA:PRIMARY> db.user.find().count()7521shardA:PRIMARY> shardB:PRIMARY> use yuhuashiswitched to db yuhuashishardB:PRIMARY> db.user.find().count()2479shardB:PRIMARY>
GAME OVER!
转载地址：http://azhji.baihongyu.com/