当前位置:网站首页>Introduction to mongodb replica set

Introduction to mongodb replica set

2020-11-14 10:00:22 osc_ 3hevjybf

mongodb Introduction to replica set

Preface :mongodb Because of the high performance 、 High availability 、 Support features such as fragmentation , As a non relational database, it is widely used . Its high availability is mainly reflected in mongodb On the replica set of ( It can be simply understood as a cluster with one master and many slaves ), This article mainly introduces from the replica set 、 Build replica sets locally 、 Let's take a look at the three aspects of replica set reading and writing data mongodb Replica set .

One 、 mongodb Introduction to replica set

mongodb Replica set (Replica Set) Including the master node (primary) With the replica node (Secondaries).

The master node can only have one , All write requests are processed on the master node . A replica node can have more than one , By synchronizing the operation log of the master node (oplog) To back up the master node data .

After the primary node is down , Replica nodes with voting rights will automatically initiate elections , And elect a new master node from it .

Replica nodes can be configured to specify their specific properties , Like elections 、 hide 、 Delay synchronization, etc , Up to 50 Replica nodes , But there can only be 7 Replica nodes can participate in the election . Although replica nodes cannot handle write operations , But it can handle read requests , I'll talk about .

It takes at least three nodes to build a replica set cluster : A master node , Two backup nodes , If the three nodes are reasonably distributed , Basically, online data can be guaranteed 99.9% Security . The architecture of the three nodes is shown in the figure below :

If there is only one master node , A replica node , And there are no resources to use as the second replica node , Then we can start an arbiter node (arbiter), No data , For election purposes only , As shown in the figure below :

When the master node is down , Then the two replica nodes will vote , Select a new master node from it , The process is as follows :

For replica set member properties , In particular, we need to explain these :priority、hidden、slaveDelay、tags、votes.

  • priority

    For replica nodes , This attribute can be used to increase or decrease the probability that the node will be elected as the master node , The value range is 0-1000( If it is arbiters, Then the value is only 0 perhaps 1), The bigger the data , The more likely it is to become the master node , If configured to 0, Then he can't be elected as the master node , And you can't initiate elections .

    This feature is usually used when there are multiple data centers , Like a master data center , A backup data center , The primary data center will be faster , If the master node fails , We certainly hope that the new master node will also be generated in the master data center , Then we can set the priority of the replica node in the backup data center to 0, As shown in the figure below :

  • hidden

    Hidden nodes synchronize data from the master node , It is not visible to the client , stay mongo shell perform db.isMaster() Method does not show the node , Hidden nodes must be Priority by 0, That is, it cannot be elected as the master node . But if you have the right to configure elections , You can vote .

    Because hidden nodes are not visible to clients , So it doesn't interact with the client , It can be used to backup data or run some back-end scheduled tasks , The details are as follows: ,4 Each backup node synchronizes data from the primary node , among 1 Hidden nodes :

  • slaveDelay

    The synchronization delay of master node is called synchronization delay , For example, delay time configuration 1 Hours , Now the time is 09:52, Then the delay node only synchronizes to the master node 08:52 Previous data . In addition, it should be noted that the delay node must be a hidden node , And Priority by 0.

    What's the use of this delay node ? Developers who have experienced database misoperation must know the answer , That's to prevent database misoperation , For example, before updating the service , Generally, the database update script will be executed first , If there is a problem with the script , And there is no backup before operation , The data may not be retrieved . But if the delay node is configured , That's the end of misoperation , And this node can cover the bottom , I can only say that this function is really intimate . The specific delay node is shown in the figure below :

  • tags

    Supports tagging replica set members , When querying data, you will use , For example, find the replica node of the corresponding tag , And then read the data from that node , This is also very useful , Nodes can be classified according to their labels , When querying data, the client of different services specifies the node of its corresponding tag , Increase or decrease the number of nodes in a label , And I'm not afraid it will affect the services that use other tags .Tags Specific use of , The following sections of the article will also talk about .

  • votes

    Indicates whether the node has permission to participate in the election , The maximum configuration is 7 Replica nodes participate in the election .

Two 、 And build the test set

install mongodb course :https://docs.mongodb.com/manual/installation/

Let's build a set of P-S-S The replica set of the structure (1 individual Primary node ,2 individual Secondary node ), The general process is : Start with three different ports mongod process , And then in mongo shell Execute the command set in the replica .

Start a single mongod The command for the instance is :

mongod --replSet rs0 --port 27017 --bind_ip localhost,<hostname(s)|ip address(es)> --dbpath /data/mongodb/rs0-0 --oplogSize 128

Parameter description :

Parameters explain Example
replSet Replica set name rs0
port mongod Instance port 27017
bind_ip The address list of the instance , Only local access can be set to localhost perhaps 127.0.0.1, Internal domain name is recommended for production environment Localhost
dbpath Data storage location /data/mongodb/rs0-0
oplogSize Operation log size 128

The construction steps are as follows :

  1. First create three directories to store the data of these three nodes

    mkdir -p /data/mongodb/rs0-0 /data/mongodb/rs0-1 /data/mongodb/rs0-2

  2. Start three... Respectively mongod process , Ports are :27018,27019,27020

    first :
    mongod --replSet rs0 --port 27018 --bind_ip localhost --dbpath /data/mongodb/rs0-0 --oplogSize 128

    the second :
    mongod --replSet rs0 --port 27019 --bind_ip localhost --dbpath /data/mongodb/rs0-1 --oplogSize 128

    Third :
    mongod --replSet rs0 --port 27020 --bind_ip localhost --dbpath /data/mongodb/rs0-2 --oplogSize 128

  3. Use mongo Enter first mongod Example , Use rs.initiate() To initialize

    Log in to 27018: mongo localhost:27018

    perform :

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19


















    rsconf = {
    _id: "rs0",
    members: [
    {
    _id: 0,
    host: "localhost:27018"
    },
    {
    _id: 1,
    host: "localhost:27019"
    },
    {
    _id: 2,
    host: "localhost:27020"
    }
    ]
    }

    rs.initiate( rsconf )


















    The above has completed the construction of a replica set , stay mongo shell In the implementation of rs.conf() You can see every node in the host、arbiterOnly、hidden、priority、 votes、slaveDelay Equal attribute , Is it super simple ..

    perform rs.conf() , The results are as follows :

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64































































    rs.conf()
    {
    "_id" : "rs0",
    "version" : 1,
    "protocolVersion" : NumberLong(1),
    "writeConcernMajorityJournalDefault" : true,
    "members" : [
    {
    "_id" : 0,
    "host" : "localhost:27018",
    "arbiterOnly" : false,
    "buildIndexes" : true,
    "hidden" : false,
    "priority" : 1,
    "tags" : {

    },
    "slaveDelay" : NumberLong(0),
    "votes" : 1
    },
    {
    "_id" : 1,
    "host" : "localhost:27019",
    "arbiterOnly" : false,
    "buildIndexes" : true,
    "hidden" : false,
    "priority" : 1,
    "tags" : {

    },
    "slaveDelay" : NumberLong(0),
    "votes" : 1
    },
    {
    "_id" : 2,
    "host" : "localhost:27020",
    "arbiterOnly" : false,
    "buildIndexes" : true,
    "hidden" : false,
    "priority" : 1,
    "tags" : {

    },
    "slaveDelay" : NumberLong(0),
    "votes" : 1
    }
    ],
    "settings" : {
    "chainingAllowed" : true,
    "heartbeatIntervalMillis" : 2000,
    "heartbeatTimeoutSecs" : 10,
    "electionTimeoutMillis" : 10000,
    "catchUpTimeoutMillis" : -1,
    "catchUpTakeoverDelayMillis" : 30000,
    "getLastErrorModes" : {

    },
    "getLastErrorDefaults" : {
    "w" : 1,
    "wtimeout" : 0
    },
    "replicaSetId" : ObjectId("5f957f12974186fc616688fb")
    }
    }































































Pay special attention to : stay mongo shell in , Yes rs Follow db.

  • rs A replica set , Yes rs.initiate(),rs.conf(), rs.reconfig(), rs.add() How to operate the replica set
  • db It's a database , Under it are some operations on the database , For example, we will use db.isMaster(), db.collection.find(), db.collection.insert() etc. .

Let's test it again Automatic Failover

  1. You can stop the master node directly localhost:27018 To test when the primary node is down , The replica node re elects a new master node , Automatic fail over (Automatic Failover)

    Kill the master node 27018 after , You can see 27019 The election section of the output log of ,27019 Launch an election , And successfully participated in the election to become the master node :

    1
    2
    3
    4
    5




    2020-10-26T21:43:58.156+0800 I  REPL     [replexec-304] Scheduling remote command request for vote request: RemoteCommand 100694 -- target:localhost:27018 db:admin cmd:{ replSetRequestVotes: 1, setName: "rs0", dryRun: false, term: 17, candidateIndex: 1, configVersion: 1, lastCommittedOp: { ts: Timestamp(1603719830, 1), t: 16 } }
    2020-10-26T21:43:58.156+0800 I REPL [replexec-304] Scheduling remote command request for vote request: RemoteCommand 100695 -- target:localhost:27020 db:admin cmd:{ replSetRequestVotes: 1, setName: "rs0", dryRun: false, term: 17, candidateIndex: 1, configVersion: 1, lastCommittedOp: { ts: Timestamp(1603719830, 1), t: 16 } }
    2020-10-26T21:43:58.159+0800 I ELECTION [replexec-301] VoteRequester(term 17) received an invalid response from localhost:27018: ShutdownInProgress: In the process of shutting down; response message: { operationTime: Timestamp(1603719830, 1), ok: 0.0, errmsg: "In the process of shutting down", code: 91, codeName: "ShutdownInProgress", $clusterTime: { clusterTime: Timestamp(1603719830, 1), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } } }
    2020-10-26T21:43:58.164+0800 I ELECTION [replexec-305] VoteRequester(term 17) received a yes vote from localhost:27020; response message: { term: 17, voteGranted: true, reason: "", ok: 1.0, $clusterTime: { clusterTime: Timestamp(1603719830, 1), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, operationTime: Timestamp(1603719830, 1) }
    2020-10-26T21:43:58.164+0800 I ELECTION [replexec-304] election succeeded, assuming primary role in term 17




  2. And then execute rs.status() View the current replica set , You can see 27019 Become the master node ,27018 The display has been hung up health = 0

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90

























































































    rs.status()
    {
    "set" : "rs0",
    "date" : ISODate("2020-10-26T13:44:22.071Z"),
    "myState" : 1,
    "heartbeatIntervalMillis" : NumberLong(2000),
    "majorityVoteCount" : 2,
    "writeMajorityCount" : 2,
    "members" : [
    {
    "_id" : 0,
    "name" : "localhost:27018",
    "ip" : "127.0.0.1",
    "health" : 0,
    "state" : 8,
    "stateStr" : "(not reachable/healthy)",
    "uptime" : 0,
    "optime" : {
    "ts" : Timestamp(0, 0),
    "t" : NumberLong(-1)
    },
    "optimeDurable" : {
    "ts" : Timestamp(0, 0),
    "t" : NumberLong(-1)
    },
    "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
    "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
    "lastHeartbeat" : ISODate("2020-10-26T13:44:20.202Z"),
    "lastHeartbeatRecv" : ISODate("2020-10-26T13:43:57.861Z"),
    "pingMs" : NumberLong(0),
    "lastHeartbeatMessage" : "Error connecting to localhost:27018 (127.0.0.1:27018) :: caused by :: Connection refused",
    "syncingTo" : "",
    "syncSourceHost" : "",
    "syncSourceId" : -1,
    "infoMessage" : "",
    "configVersion" : -1
    },
    {
    "_id" : 1,
    "name" : "localhost:27019",
    "ip" : "127.0.0.1",
    "health" : 1,
    "state" : 1,
    "stateStr" : "PRIMARY",
    "uptime" : 85318,
    "optime" : {
    "ts" : Timestamp(1603719858, 1),
    "t" : NumberLong(17)
    },
    "optimeDate" : ISODate("2020-10-26T13:44:18Z"),
    "syncingTo" : "",
    "syncSourceHost" : "",
    "syncSourceId" : -1,
    "infoMessage" : "",
    "electionTime" : Timestamp(1603719838, 1),
    "electionDate" : ISODate("2020-10-26T13:43:58Z"),
    "configVersion" : 1,
    "self" : true,
    "lastHeartbeatMessage" : ""
    },
    {
    "_id" : 2,
    "name" : "localhost:27020",
    "ip" : "127.0.0.1",
    "health" : 1,
    "state" : 2,
    "stateStr" : "SECONDARY",
    "uptime" : 52468,
    "optime" : {
    "ts" : Timestamp(1603719858, 1),
    "t" : NumberLong(17)
    },
    "optimeDurable" : {
    "ts" : Timestamp(1603719858, 1),
    "t" : NumberLong(17)
    },
    "optimeDate" : ISODate("2020-10-26T13:44:18Z"),
    "optimeDurableDate" : ISODate("2020-10-26T13:44:18Z"),
    "lastHeartbeat" : ISODate("2020-10-26T13:44:20.200Z"),
    "lastHeartbeatRecv" : ISODate("2020-10-26T13:44:21.517Z"),
    "pingMs" : NumberLong(0),
    "lastHeartbeatMessage" : "",
    "syncingTo" : "localhost:27019",
    "syncSourceHost" : "localhost:27019",
    "syncSourceId" : 1,
    "infoMessage" : "",
    "configVersion" : 1
    }
    ]
    }

























































































  3. Start again 27018:
    mongod --replSet rs0 --port 27018 --bind_ip localhost --dbpath /data/mongodb/rs0-0 --oplogSize 128

    It can be at the node 27019 We can see in the log that 27018, And has become a replica node , adopt rs.status The same is true of the results .

    1
    2020-10-26T21:52:06.871+0800 I  REPL     [replexec-305] Member localhost:27018 is now in state SECONDARY

3、 ... and 、 Some features of copyset writing and reading

Write about (Write concern)

Replica set write focus refers to writing a piece of data , After the master node completes processing , After the other replica nodes bearing the data also confirm that the write is successful , In order to return the success of writing data to the client .

The main function of the main node is to solve this problem , Data has not yet been synchronized to the replica node , The problem of data loss .

The number of nodes can be configured , The default configuration {“w”:1}, In this way, it means that the master node writes data successfully and returns success to the client ,“w” Configure to 2, In addition to the master node , You also need to receive one of the replica nodes to return a write success ,“w” It can also be configured as “majority”, Indicates that most of the nodes in the cluster that need to carry data and have voting rights return to write successfully .

As shown in the figure below ,P-S-S structure ( One primary node , Two secondary node ), Write a request with w : “majority” , Then after the master node has finished writing , Data synchronization to the first replica node , And after the first replica node writes data successfully , To return success to the client .

About writing, focus on how to operate in practice , There are two ways :

  1. Specify in the write request writeConcern Related parameters , as follows :

    1
    2
    3
    4



    db.products.insert(
    { item: "envelopes", qty : 100, type: "Clasp" },
    { writeConcern: { w: "majority" , wtimeout: 5000 } }
    )



  2. Modify replica set getLastErrorDefaults To configure , as follows :

    1
    2
    3


    cfg = rs.conf()
    cfg.settings.getLastErrorDefaults = { w: "majority", wtimeout: 5000 }
    rs.reconfig(cfg)


Reading preference (Read preference)

Reading is not the same as writing , For consistency , Write only through the master node , But read can choose the master node , The replica node can also be selected , The difference is that the master data is up to date , Replica nodes may have delays due to synchronization problems , But reading data from the replica node can disperse the pressure on the primary node .

Because there will be multiple nodes carrying data , How does the client choose to read from that node ? There are mainly 3 Conditions (Tag Sets、 maxStalenessSeconds、Hedged Read),5 Patterns (primary、primaryPreferred、secondary、secondaryPreferred、nearest)

First of all 5 Patterns , Its characteristics are shown in the table below :

Pattern characteristic
primary All read requests are read from the master node
primaryPreferred The master node is normal , All read requests are read from the master node , If the master node fails , Read from the eligible replica nodes
secondary All read requests are read from the replica node
secondaryPreferred All read requests are read from the replica node , But if the replica nodes all hang up , Then read from the master node
nearest It's mainly about network latency , Select the node with the least latency , Both the master node and the replica node can

Besides, under the 3 Conditions , The condition is that on the basis of conforming to the pattern , Then according to the conditions to delete the specific node

  1. Tag Sets( label )

    seeing the name of a thing one thinks of its function , This can label nodes , And when you look up the data , You can select the corresponding node according to the label , Then look up the data in this node . Can pass mongo shell Use rs.conf() View the current node under the tags, Modify or add tags The process is the same as the above modification getLastErrorDefaults To configure , Such as :cfg.members[n].tags = { "region": "South", "datacenter": "A" }

  2. maxStalenessSeconds ( Maximum tolerable synchronization delay )

    seeing the name of a thing one thinks of its function +1, This value refers to the time when the replica node synchronizes the primary write Follow The comparison value of the actual latest write time of the master node , If the master node fails , Compare it with the latest write time in the replica set .

    This value suggests setting , Avoid not synchronizing the primary node data for a long time due to some replica node network reasons , And then I read older data . In particular, this value needs to be set 90s above , Because the client is timing to check the synchronization delay time of replica node , The data won't be particularly accurate , Set up than 90s Small , It throws an exception .

  3. Hedged Read ( Offset read )

    This option is in fragmentation cluster MongoDB 4.4 Supported only after version , finger mongos When an instance routes a read request, it is sent to two eligible replica set nodes at the same time , Then the one who returns the result first returns the result to the client .

That's the question , How can such an easy-to-use pattern and condition be used in query requests ?

  1. Connecting to a database in code , Use connection string uri when , You can add the following three parameters

    | Parameters | explain |
    | ——————- | ———————————————————— |
    | readPreference | Pattern , Enumeration values are :primary( The default value is )、 primaryPreferred、secondary、secondaryPreferred、nearest |
    | maxStalenessSeconds | Maximum synchronization delay seconds , Value 0 - 90 Will report a mistake , -1 There is no maximum |
    | readPreferenceTags | label , If the label is { “dc”: “ny”, “rack”: “r1” }, It's in uri by readPreferenceTags=dc:ny,rack:r1 |



    For example, the following :

    mongodb://db0.example.com,db1.example.com,db2.example.com/?replicaSet=myRepl&readPreference=secondary&maxStalenessSeconds=120&readPreferenceTags=dc:ny,rack:r1

  1. stay mogo shell in , have access to cursor.readPref() perhaps Mongo.setReadPref()

    cursor.readPref() The parameters are : mode、tag set、hedge options, The specific request is as follows

    1
    2
    3
    4
    5




    db.collection.find({ }).readPref(
    "secondary", // mode
    [ { "datacenter": "B" }, { } ], // tag set
    { enabled: true } // hedge options
    )




    Mongo.setReadPref() similar , It's just setting the request conditions in advance , That way, you don't have to put... At the end of every request readPref Conditions .

You can simply test this function in the built cluster

  1. Log in to the master node : mongo localhost:27018

  2. Insert a piece of data : db.nums.insert({name: “num0”})

    Query in the current node : db.nums.find()

    You can see this data : { "_id" : ObjectId("5f958687233b11771912ced5"), "name" : "num0" }

  3. Log in to the replica node : mongo localhost:27019

    Inquire about :db.nums.find()

    Because the default query mode is primary, Therefore, an error will be reported when querying in the replica node , as follows :

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14













    Error: error: {
    "operationTime" : Timestamp(1603788383, 1),
    "ok" : 0,
    "errmsg" : "not master and slaveOk=false",
    "code" : 13435,
    "codeName" : "NotMasterNoSlaveOk",
    "$clusterTime" : {
    "clusterTime" : Timestamp(1603788383, 1),
    "signature" : {
    "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
    "keyId" : NumberLong(0)
    }
    }
    }













    When querying, the specified mode is “secondary”: db.nums.find().readPref(“secondary")

    You can query the inserted data :{ "_id" : ObjectId("5f958687233b11771912ced5"), "name" : "num0" }

Conclusion

All of the above is reading MongoDB After the official documents , Then pick some simple and important points to make a summary , If you're right MongoDB Interested in , It is suggested to have a direct bite Official documents .

Experience sharing
Thank you for reading , This paper is written by Bump lab copyright . If you reprint , Please indicate the source : Bump lab ( https://aotu.io/notes/2020/11/12/mongo-replica-set/
Last update :2020-11-13 14:52:50

版权声明
本文为[osc_ 3hevjybf]所创,转载请带上原文链接,感谢

随机推荐