当前位置:网站首页>Construction of high concurrency architecture (2)

Construction of high concurrency architecture (2)

2021-06-23 23:19:59 Fairy tale tells my ending

One 、Ceph Swift API Interface development

Swift By Rackspace A project developed to provide scalable storage for Cloud Computing . Focus on object storage , And provide a set of REST Style Api To visit ,  And Ceph Strong consistency is different ,  It's the ultimate consistency . Both are excellent open source projects ,  There is no obvious distinction between good and bad , It's different in use scenarios ,  If it's focused on object storage ,  Then you can choose swift To meet the needs ,  If there are block storage requirements ,  So choose Ceph More appropriate . Choose here Ceph,  Because the gateway can be adapted and compatible swift api,  At the same time, it has strong scalability in data access .

1.1 preparation

establish Swift user , For interface request authentication , stay 140 To execute the following order

sudo radosgw-admin user create --subuser="cephtester:subtester" -- uid="cephtester" --display-name="cephtester" --key-type=swift --secret="ljx" - 
-access=full

uid Primary users , subuser For subuser information , secret Specify the key , If not specified, it will be generated randomly , access Have permission settings , In the code, use... In the return information user and secret_key.

 

{
    "user_id": "cephtester",
    "display_name": "cephtester",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [
        {
            "id": "cephtester:subtester",
            "permissions": "full-control"
        }
    ],
    "keys": [],
    "swift_keys": [
        {
            "user": "cephtester:subtester",
            "secret_key": "ljx"
        }
    ],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

 

Create administrator account :

radosgw-admin user create --uid=mgruser --display-name=mgruser --system
{
    "user_id": "mgruser",
    "display_name": "mgruser",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [],
    "keys": [
        {
            "user": "mgruser",
            "access_key": "KI0XONO70L11Q9XGRE3C",
            "secret_key": "RHkmPPMvkvy10mi4kmJA3vStJe2bmcoArYYsDwn6"
        }
    ],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "system": "true",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

According to the generated access_key And secret_key, perform : These two values are generated above

 

ceph dashboard set-rgw-api-access-key KI0XONO70L11Q9XGRE3C 
ceph dashboard set-rgw-api-secret-key RHkmPPMvkvy10mi4kmJA3vStJe2bmcoArYYsDwn6

 

Open the management interface ,http://192.168.32.140:8443/#/rgw/user  You can see the two users we just created :

 

 

1.2、 File server setup

Build a separate Maven engineering , It is specially used to upload and download files , The engineering coordinates are as follows :

 

 POM.XML

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>spring-cloud-service</artifactId>
        <groupId>com.ghy</groupId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.ghy</groupId>
    <artifactId>spring-cloud-file-service</artifactId>

    <description>
         File upload service 
    </description>
    <dependencies>
        <!-- Rados Java Api rely on  -->
        <dependency>
            <groupId>com.ceph</groupId>
            <artifactId>rados</artifactId>
            <version>0.6.0</version>
        </dependency>
        <!-- Cephfs  The file system depends on  -->
        <dependency>
            <groupId>com.ceph</groupId>
            <artifactId>libcephfs</artifactId>
            <version>0.80.5</version>
        </dependency>
        <!--swift-->
        <dependency>
            <groupId>org.javaswift</groupId>
            <artifactId>joss</artifactId>
            <version>0.10.2</version>
        </dependency>
    </dependencies>
</project>

bootstrap.yml

server:
  port: 8082
spring:
  application:
    name: spring-cloud-file-service
  cloud:
    nacos:
      config:
        file-extension: yaml
        server-addr: 192.168.32.197:8848
      discovery:
        #Nacos Registered address of 
        server-addr: 192.168.32.197:8848

ceph:
  username: cephtester:subtester  #Ceph To configure   Primary user name : Subuser name 
  password: ljx # Secret key 
  authUrl: http://192.168.32.195:8443/auth/1.0 # Interface access path 
  defaultContainerName: user_datainfo # Default container name ( It doesn't matter if you can write by yourself )
# Picture path 
cephurl: http://localhost:8082/file/download/

# Log configuration 
logging:
  pattern:
    console: "%msg%n"

Next, create a ContainerConfig Class to complete the ceph Initialization of

@Data
@Configuration
@ConfigurationProperties(prefix = "ceph")
public class ContainerConfig {

    private String username;
    private String password;
    private String authUrl;
    private String defaultContainerName;


    /*****
     * 1、 Create account information 
     */
    @Bean
    public Account account(){
        AccountConfig config = new AccountConfig();
        config.setUsername(username);
        config.setPassword(password);
        config.setAuthUrl(authUrl);
        config.setAuthenticationMethod(AuthenticationMethod.BASIC);
        return new AccountFactory(config).createAccount();
    }


    /*****
     * 2、 Create container objects 
     */
    @Bean
    public Container container(){
        Container container = account().getContainer(defaultContainerName);
        if(!container.exists()){
            return  container.create();
        }
        return container;
    }

}
Create a file upload and download tool class FileHandler , The code is as follows :
@Component
public class FileHandler {

    @Autowired
    private Container container;

    /****
     *  Upload files 
     */
    public void upload(String filename,byte[] buffer){
        // Get the container 
        StoredObject object = container.getObject(filename);
        // Upload files 
        object.uploadObject(buffer);
    }


    /***
     *  File download 
     */
    public byte[] download(String filename){
        // Get the container 
        StoredObject object = container.getObject(filename);
        // download 
        byte[] bytes = object.downloadObject();
        return bytes;
    }
}
Controller creation :FileController class
@RestController
@RequestMapping(value = "/file")
public class FileController {

    @Autowired
    private FileHandler fileHandler;

    /****
     *  Upload files 
     * @param file
     * @return
     */
    @PostMapping(value = "/upload")
    public RespResult upload(MultipartFile file) throws IOException {
        // call 
        fileHandler.upload(file.getOriginalFilename(),file.getBytes());
        return RespResult.ok();
    }

    /****
     *  File download 
     * @return
     */
    @GetMapping(value = "/download/{filename}")
    public void download(@PathVariable String filename, HttpServletResponse response) throws IOException {
        // call 
        byte[] buffer = fileHandler.download(filename);
        ServletOutputStream os = response.getOutputStream();
        os.write(buffer);
        os.flush();
        os.close();
    }
}
Create startup class :SpringCloudFileServiceApplication
@SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
public class SpringCloudFileServiceApplication {

    public static void main(String[] args) {
        SpringApplication.run(SpringCloudFileServiceApplication.class,args);
    }
}
File upload test http://localhost:8082/file/upload

 

 

File download test :http://localhost:8082/file/download/1624446315(1).jpg

 

 

 

版权声明
本文为[Fairy tale tells my ending]所创,转载请带上原文链接,感谢
https://chowdera.com/2021/06/20210623231925138W.html

随机推荐