当前位置:网站首页>Design of FTP client with more than one point

Design of FTP client with more than one point

2020-12-06 23:24:54 havealex

Project background

FileZilla FTP Client This software can only manually upload files to FTP, It seems that I haven't found a regular scan of a local directory , And then perform the function of regular upload . One of the problems recently is : Deployed on the server to many FTP Client timer program , Every FTP client exe Executable functions are similar , All of them will conform to the file rules under a folder in the local server ( Such as *.json,*.xml) File by FTP Upload to specified FTP In a directory on the server . But there are too many programs , In this way, if you need to upload more than one radar file to multiple FTP On the server , It will open multiple FTP Push client program , In this way, the server will generate multiple exe Executable program . As shown in the figure below :  The server FTP Push client It doesn't look good , And it's hard to tell which one is FTP Program , To solve this problem , It is best to FTP Push the client to make a little more than one .

FTP File Upload Version1.0 edition

About uploading only one FTP Server address MFC The program can refer to the blog I wrote before :VC++ libcurl FTP Upload client program

I've uploaded the source code to Github and Gitee The above , The use of C/C++ Library has libcurlboostpugixml

FTPUpload-Github Address FTPUpload-Gitee Address FTPUpload It's based on MFC Of FTP Push client program , Used libcurl Realization FTP push , Use pugixml Realization xml Read and write configuration files , Also used. Boost Library is used for conversion of directory rules ( Relating to date ). The flow chart of program execution is shown in the figure below :  Program flow chart

The procedure is shown in the figure below : FTP File Upload

FTP File Upload Version2.0 edition

On the basis of the first version, we made another version , Added Windows Service program FTP File Upload Version2.0 edition

FTP File Upload Version3.0 edition

Function description :

  • FTP Multiple data sources FTP Target source push client implementation ( Support a little more ):
  • 1. Get task information from database , Get the information of the local file source to be uploaded separately ( Include local directories that need to be scanned 、 Directory rules 、 File rules, etc ) as well as
  • FTP Destination information ( Include FTP Remote URL Address 、FTP user name 、 password 、 Remote directory address, etc )
  • 2、 Traverse task list
  • 3、 For each task , Traverse the file source ( There may be more than one )
  • 4、 For each file source , Traversal needs to upload FTP Address ( There may be more than one )
  • 5、 For each file source and for some FTP Data upload channel of destination information , Create a FTP Upload thread
  • 6、 For a single FTP Upload thread , perform FTP Upload , And write the successful or failed upload to the database , In order to repeat transmission many times ( Here, we should pay attention to exclude the files that do not exist in the local directory but still have upload records in the database )

Program execution flow chart

The program flow chart is shown in the following figure : FTP Multi source and multi destination push client - flow chart

Database design

The database I use is Sqlite3 Library , It was chosen mainly because it was light and didn't need to be installed in advance . It mainly involves three tables :t_task Task list 、t_source Data source table 、t_dest namely FTP Destination surface , It's better to add a list of uploaded files according to your needs , To prevent the same file from being uploaded repeatedly . The structure of the three tables is as follows :

  • surface 1 t_task task

  • surface 2 t_source source

  • surface 3 t_dest dest

Of course , You can choose Sqlite3 Database as data source , You can also use xml Configuration file as data source .

For the convenience of my use Sqlite3 database , Easy and convenient , Used CodeProject The above one is about Windows Under the system SQLite Of C++ Wrapper class CppSQLite - C++ Wrapper for SQLite, Based on this kind of simple encapsulation based on Sqlite3 The operation of . The core code is as follows :

// FtpUploadMulti.cpp :  This file contains  "main"  function . Program execution will start and end here .
//

#include <iostream>
#include "TaskDB.h"
#include "FTPUpload.h"
#include <thread>

using namespace std;

/**
 *  Function description :
 * FTP Multiple data sources FTP Target source push client implementation ( Support a little more ):
 * 1. Get task information from database , Get the information of the local file source to be uploaded separately ( Include local directories that need to be scanned 、 Directory rules 、 File rules, etc ) as well as 
 * FTP Destination information ( Include FTP Remote URL Address 、FTP user name 、 password 、 Remote directory address, etc )
 * 2、 Traverse task list 
 * 3、 For each task , Traverse the file source ( There may be more than one )
 * 4、 For each file source , Traversal needs to upload FTP Address ( There may be more than one )
 * 5、 For each file source and for some FTP Data upload channel of destination information , Create a FTP Upload thread 
 * 6、 For a single FTP Upload thread , perform FTP Upload , And write the successful or failed upload to the database , In order to repeat transmission many times ( Here, we should pay attention to exclude the files that do not exist in the local directory but still have upload records in the database )
 */
// FTP Upload thread function 
void ftpUploadRun(FTPInfo item)
{
	FTPUpload ftpUpload;
	//  Set source and FTP Destination information 
	//  Set the local file directory 、 Scanned directory rules 、 Document rule 
	ftpUpload.set_local_path(item.localPath.c_str(), item.folderDir.c_str(),
		item.localFile.c_str());
	//  Set up FTP Uploaded address 、 User name and password 
	ftpUpload.set_remote_path(item.remoteUrl.c_str(),
		item.remoteUser.c_str(), item.remotePwd.c_str());
	//  perform FTP Upload 
	ftpUpload.upload();
}

int main()
{
    //  Read the task list 
   
    //  Traverse task list , Get the file source and FTP Destination 

	//  Timing execution FTP Upload task 
	//  You can create one for each task FTP Upload thread , Or for each channel ( Upload object ) Create a FTP Upload thread 
	signal_log lg;
	TaskDB taskDB(lg);
	TaskItemArray taskList;
	//TaskItem item;
	taskDB.init();
	taskList = taskDB.getTaskList();
	//  Traverse task list 
	std::vector<std::thread> thrVec;

	for (auto taskIter = taskList.begin(); taskIter != taskList.end(); taskIter++)
	{
		//  Get source sid
		int sid = taskIter->sourceId;
		//  obtain FTP Destination did
		int did = taskIter->destId;
		//  according to sid Get the source list 
		SourceItemArray srcList;
		srcList = taskDB.getSourceListById(sid);
		printf("\n---------------  The source of information  ---------------\n");
		for (auto srcIter = srcList.begin(); srcIter != srcList.end(); srcIter++)
		{
			printf("sid: %d, cid: %d, localPath: %s, folderRule: %s, fileRule: %s\n",
				srcIter->sid, srcIter->cid, srcIter->localPath.c_str(), srcIter->folderRule.c_str(), srcIter->fileRule.c_str());
		}
		//  according to did obtain FTP Destination information list 
		DestItemArray destList;
		destList = taskDB.getDestListById(did);
		printf("\n--------------- FTP Destination information  ---------------\n");
		for (auto destIter = destList.begin(); destIter != destList.end(); destIter++)
		{
			printf("did: %d, cid: %d, ftpUrl: %s, ftpFolder: %s, enableFolder: %s, ftpUser: %s, ftpPwd: %s\n",
				destIter->did, destIter->cid, destIter->ftpUrl.c_str(), destIter->ftpFolder.c_str(),
				destIter->enableFolder ? "true" : "false", destIter->ftpUser.c_str(), destIter->ftpPwd.c_str());
		}

		//  Get the total number of threads ( namely FTP Total number of data upload channels : Number of sources  *  Destination data )
		int threadNum = srcList.size() * destList.size();
		printf("FTP Total number of data upload channels ( Total number of threads ): %d\n", threadNum);

		for (auto srcIter = srcList.begin(); srcIter != srcList.end(); srcIter++)
		{
			SourceItem srcItem = *srcIter;
			for (auto destIter = destList.begin(); destIter != destList.end(); destIter++)
			{
				DestItem destItem = *destIter;

				FTPInfo item;
				item.uploadRate = 5;
				item.localPath = srcItem.localPath;
				item.folderDir = srcItem.folderRule;
				item.localFile = srcItem.fileRule;
				item.remoteUrl = destItem.ftpUrl;
				item.remoteUser = destItem.ftpUser;
				item.remotePwd = destItem.ftpPwd;

				//  Create a FTP Upload thread , And add it to the thread list 
				thrVec.push_back(std::thread(ftpUploadRun, item));
			}
		}
	}

	/*while (true)
	{
		msleep(10);
	}*/
	getchar();
	for (auto &thr : thrVec)
	{
		if (thr.joinable())
		{
			thr.join();
		}
	}

	return 0;
}

Version3.0

版权声明
本文为[havealex]所创,转载请带上原文链接,感谢
https://chowdera.com/2020/12/202012062322299725.html