当前位置:网站首页>Handwritten koa static source code, in-depth understanding of static server principle

Handwritten koa static source code, in-depth understanding of static server principle

2020-12-07 12:39:53 A kind of Jiang Pengfei

This article continues with the previous Koa Source series , There are already two articles in this series :

  1. The first one explains Koa Core architecture and source code : Handwriting Koa.js Source code
  2. The second one explains it @koa/router Architecture and source code : Handwriting @koa/router Source code

This article will continue to talk about a commonly used middleware ----koa-static, This middleware is used to build a static server .

In fact, before me Use Node.js Native API Write a web The server I've talked about how to return a static file , The code is ugly , The basic process is similar :

  1. Get the correct file address through the request path
  2. Get the corresponding file through the address
  3. Use Node.js Of API Return the corresponding file , And set the corresponding header

koa-static The code is more general , More elegant , And better support for big files , Let's see how he did it . This article still uses the consistent routine , Let's take a look at his basic usage , Then read the source code from the basic usage , A version of his handwriting to simplify the source .

Basic usage

koa-static It's easy to use , The main code is just one line :

const Koa = require('koa');
const serve = require('koa-static');

const app = new Koa();

//  This is the main line of code 
app.use(serve('public'));

app.listen(3001, () => {
    console.log('listening on port 3001');
});

In the above code serve Namely koa-static, It will return a Koa middleware , then Koa The instance of the middleware can directly refer to the middleware .

serve Method supports two parameters , The first is the directory of static files , The second parameter is some configuration items , Can not pass . Like the code above serve('public') It means public The files under the folder can be accessed externally . For example, I put a picture in it :

image-20201125163558774

That's how you run :

image.png

Note that the path above requests /test.jpg, There's no public, explain koa-static The request path is judged , If the file is found, it is mapped to the server public Below directory , This prevents external users from discovering the server directory structure .

Handwritten source code

Back to a Koa middleware

We see koa-static What is derived is a method serve, This method should return a Koa middleware , such Koa To quote him , So let's write about this structure first :

module.exports = serve;   //  Exported is serve Method 

// serve Take two parameters 
//  The first parameter is the path address 
//  The second is configuration options 
function serve(root, opts) {
    //  Return a method , This method is consistent with koa Definition of middleware 
    return async function serve(ctx, next) {
        await next();
    }
}

call koa-send Return file

Now the middleware is empty , In fact, what he should do is return the document to , The function of returning files is also extracted into a library ----koa-send, We will see his source code later , Let's use it directly here .

function serve(root, opts) {
    //  If the effect of this line of code is 
    //  If not passed opts,opts It's an empty object {}
    //  At the same time, set its prototype to null
    opts = Object.assign(Object.create(null), opts);

    //  take root Resolve to a legal path , And on the opts Up 
    //  because koa-send The receiving path is in opts On 
    opts.root = resolve(root);
  
  	//  This is for folder compatibility , If the request path is a folder , By default, go and get index
    //  If the user does not configure index, Default index Namely index.html
    if (opts.index !== false) opts.index = opts.index || 'index.html';

  	//  Whole serve The return value of the method is a koa middleware 
  	//  accord with koa Middleware paradigm : (ctx, next) => {}
    return async function serve(ctx, next) {
        let done = false;    //  This variable marks whether the file returned successfully 

        //  Only HEAD and GET Respond to a request 
        if (ctx.method === 'HEAD' || ctx.method === 'GET') {
            try {
                //  call koa-send Send a file 
                //  If the transmission is successful ,koa-send Will return to the path , Assign a value to done
                // done Convert to bool The value is true
                done = await send(ctx, ctx.path, opts);
            } catch (err) {
                //  If not 404, It could be some 400,500 This unexpected mistake , Throw it out 
                if (err.status !== 404) {
                    throw err
                }
            }
        }

        //  adopt done To detect whether the file was sent successfully 
        //  If it doesn't work , Let the subsequent middleware continue to process it 
        //  If it works , That's the end of the request 
        if (!done) {
            await next()
        }
    }
}

opt.defer

defer It's a configuration option opt An optional parameter in it , He's a little special , The default is false, If you pass on true,koa-static It will make other middleware respond first , Even if other middleware is written in koa-static He will be asked to respond first , Finally, I respond . To achieve this , It's actually a control call next() The timing of . stay speak Koa The article of source code has already mentioned , call next() In fact, it is calling the middleware behind , So the last call is like the code above next(), That is to execute first koa-static Then execute other middleware . If you give defer Yes true, In fact, it is to execute first next(), And then execute koa-static The logic of , According to this idea, let's support defer Well :

function serve(root, opts) {
    opts = Object.assign(Object.create(null), opts);

    opts.root = resolve(root);

    //  If defer by false, Just use the logic before , Last call next
    if (!opts.defer) {
        return async function serve(ctx, next) {
            let done = false;    

            if (ctx.method === 'HEAD' || ctx.method === 'GET') {
                try {
                    done = await send(ctx, ctx.path, opts);
                } catch (err) {
                    if (err.status !== 404) {
                        throw err
                    }
                }
            }

            if (!done) {
                await next()
            }
        }
    }

    //  If defer by true, First call next, And then execute your own logic 
    return async function serve(ctx, next) {
        //  First call next, Execute the following middleware 
        await next();

        if (ctx.method !== 'HEAD' && ctx.method !== 'GET') return

        //  If ctx.body Have a value , perhaps status No 404, Indicates that the request has been processed by other middleware , I'm going straight back 
        if (ctx.body != null || ctx.status !== 404) return // eslint-disable-line

        // koa-static Their own logic is still the same , All calls koa-send
        try {
            await send(ctx, ctx.path, opts)
        } catch (err) {
            if (err.status !== 404) {
                throw err
            }
        }
    }
}

koa-static Source code on a total of dozens of lines :https://github.com/koajs/static/blob/master/index.js

koa-send

Above we see koa-static It's actually packaged koa-send, The real operation of sending files is in koa-send Inside . At the beginning of the article koa-static None of them are dry , Throw it all to koa-send 了 , That is to say, he should finish all these things :

  1. Get the correct file address through the request path
  2. Get the corresponding file through the address
  3. Use Node.js Of API Return the corresponding file , And set the corresponding header

because koa-send There's not much code , I just wrote comments in the code , Through the use of the front , We already know that his form of use is :

send (ctx, path, opts)

He takes three parameters :

  1. ctx: Namely koa The context of ctx.
  2. pathkoa-static The message is ctx.path, seen koa Source code analysis should know , This value is actually req.path
  3. opts: Some configuration items ,defer We talked about that before , Will affect the execution order , There are other cache controls .

Let's just write a send Methods! :

const fs = require('fs')
const fsPromises = fs.promises;
const { stat, access } = fsPromises;

const {
    normalize,
    basename,
    extname,
    resolve,
    parse,
    sep
} = require('path')
const resolvePath = require('resolve-path')

//  export send Method 
module.exports = send;

// send Method implementation 
async function send(ctx, path, opts = {}) {
    //  First parse the configuration item 
    const root = opts.root ? normalize(resolve(opts.root)) : '';  //  there root It's the static file directory that we configure , such as public
    const index = opts.index;    //  When you request a folder , Will read this index file 
    const maxage = opts.maxage || opts.maxAge || 0;     //  Namely http Cache control Cache-Control the maxage
    const immutable = opts.immutable || false;   //  It's also Cache-Control Cache controlled 
    const format = opts.format !== false;   // format The default is true, Used to support /directory This one doesn't / Folder request for 

    const trailingSlash = path[path.length - 1] === '/';    //  have a look path Is the ending /
    path = path.substr(parse(path).root.length)             //  Get rid of path At the beginning /

    path = decode(path);      //  In fact, that is decodeURIComponent, decode The auxiliary method is in the back 
    if (path === -1) return ctx.throw(400, 'failed to decode');

    //  If the / ending , It must be a folder , take path Change to the default file under the folder 
    if (index && trailingSlash) path += index;

    // resolvePath You can combine a root path and the relative path of a request into an absolute path 
    //  And prevent some common attacks , such as GET /../file.js
    // GitHub Address :https://github.com/pillarjs/resolve-path
    path = resolvePath(root, path)

    //  use fs.stat Get basic information about the file , By the way, check whether the file exists 
    let stats;
    try {
        stats = await stat(path)

        //  If it's a folder , also format by true, Spell it index file 
        if (stats.isDirectory()) {
            if (format && index) {
                path += `/${index}`
                stats = await stat(path)
            } else {
                return
            }
        }
    } catch (err) {
        //  Error handling , If the file doesn't exist , return 404, Otherwise return to 500
        const notfound = ['ENOENT', 'ENAMETOOLONG', 'ENOTDIR']
        if (notfound.includes(err.code)) {
          	// createError come from http-errors library , You can quickly create HTTP Error object 
            // github Address :https://github.com/jshttp/http-errors
            throw createError(404, err)
        }
        err.status = 500
        throw err
    }

    //  Set up Content-Length Of header
    ctx.set('Content-Length', stats.size)

    //  Set cache control header
    if (!ctx.response.get('Last-Modified')) ctx.set('Last-Modified', stats.mtime.toUTCString())
    if (!ctx.response.get('Cache-Control')) {
        const directives = [`max-age=${(maxage / 1000 | 0)}`]
        if (immutable) {
            directives.push('immutable')
        }
        ctx.set('Cache-Control', directives.join(','))
    }

    //  Set the return type and return content 
   if (!ctx.type) ctx.type = extname(path)
    ctx.body = fs.createReadStream(path)

    return path
}

function decode(path) {
    try {
        return decodeURIComponent(path)
    } catch (err) {
        return -1
    }
}

The above code doesn't have much complicated logic , First spell a complete address , And then use fs.stat Get basic information about the file , If the file doesn't exist , This API It's a mistake , Go straight back to 404. If the file exists , Just use fs.stat Get the information set Content-Length And some cache controlled header.

koa-send Source code is only one file , A hundred lines of code :https://github.com/koajs/send/blob/master/index.js

ctx.type and ctx.body

We can see that the above code does not directly return the file , It's just set up ctx.type and ctx.body These two values end up , Why are these two values set , The file is returned automatically ? You know the principle , We need to combine Koa Look at the source code .

I said before Koa I mentioned the source code , He expanded Node Native res, And give it inside type Attribute adds a set Method :

set type(type) {
  type = getType(type);
  if (type) {
    this.set('Content-Type', type);
  } else {
    this.remove('Content-Type');
  }
}

The purpose of this code is when you give ctx.type When setting the value , Will automatically give Content-Type Set the value ,getType It's actually another third-party library cache-content-type, It can be based on the type of file you pass in , Return matched MIME type. I just saw koa-static Source code , After searching for a long time, I didn't find where it was set up Content-Type, It turns out that it was in Koa Source code . So set up ctx.type In fact, it's set up Content-Type.

koa Extended type The attributes are shown here :https://github.com/koajs/koa/blob/master/lib/response.js#L308

I said before Koa I also mentioned the source code , When all middleware runs out , Finally, a method will be run respond To return results , In that article ,respond It's a simplified version , Direct use res.end Returned the result :

function respond(ctx) {
  const res = ctx.res; //  Take out res object 
  const body = ctx.body; //  Take out body

  return res.end(body); //  use res return body
}

Direct use res.end The return result is only suitable for some simple small objects , For example, string or something . For complex objects , Such as file , This is the right one , Because if you want to use res.write perhaps res.end Return file , You need to read the entire file into memory first , Then passed as a parameter , If the file is large , The server memory may burst . How to deal with that ? go back to koa-send Source code , We give ctx.body The set value is actually a readable stream :

ctx.body = fs.createReadStream(path)

How does this flow return ? Actually Node.js There is good support for the return flow itself . To return a value , Need to use http In the callback function res, This res In fact, it is also a stream . You can turn over again Node.js Official documents , there res It's actually http.ServerResponse An instance of a class , and http.ServerResponse It is inherited from Stream class :

image-20201203154324281

therefore res It's a stream in itself Stream, that Stream Of API You can use it .ctx.body It's using fs.createReadStream Created , So it's a readable stream , Read stream has a very convenient API You can flow content directly to a writable stream :readable.pipe, Use this API,Node.js Will automatically push the contents of the read stream to the writable stream , Data streams are automatically managed , So even if the read stream is faster , The target writable stream is not overloaded , And even if your files are big , Because it's not read into memory at once , It's streaming in , So it won't explode . So we are Koa Of respond It's for the dirty body That's it :

function respond(ctx) {
  const res = ctx.res; 
  const body = ctx.body; 
  
  //  If body It's a stream , Direct use pipe Bind it to res On 
  if (body instanceof Stream) return body.pipe(res);

  return res.end(body); 
}

Koa Source code for the processing of the flow here :https://github.com/koajs/koa/blob/master/lib/application.js#L267

summary

Now? , We can write our own koa-static To replace the official , The running effect is the same . Finally, let's review the main points of this article :

  1. This article is about Koa Commonly used static service middleware koa-static Source code analysis .

  2. Because it's a Koa Middleware , therefore koa-static The return value of is a method , And it needs to conform to the middleware paradigm : (ctx, next) => {}

  3. As a static service middleware ,koa-static The following things should have been done :

    1. Get the correct file address through the request path
    2. Get the corresponding file through the address
    3. Use Node.js Of API Return the corresponding file , And set the corresponding header

    But he didn't do any of these things , Throw them all koa-send 了 , So his official documents say he's just wrapper for koa-send.

  4. As a wrapper He also supports a special configuration item opt.defer, This configuration item can control it in all of Koa Execution timing in middleware , In fact, it's called next The timing of . If you pass this parameter true, He calls first next, Let other middleware execute first , Finally, execute it yourself , vice versa . With this parameter , You can take /test.jpg This request is first treated as a normal route , Try static file again if route doesn't match , This is useful in some situations .

  5. koa-send It's really about dealing with static files , He did all the three things mentioned earlier , In the splicing file path also used resolvePath To defend against common attacks .

  6. koa-send The file was retrieved using fs Modular API Created a readable stream , And assign it to ctx.body, Also set up ctx.type.

  7. adopt ctx.type and ctx.body Returning to the requester is not koa-send The function of , It is Koa Its own function . because http Modules provide and res It's a writable stream in itself , So we can read the stream of pipe The function will directly ctx.body Bound to the res On , The rest of the work Node.js Will automatically help us finish .

  8. Use stream (Stream) There are several advantages to reading and writing files :

    1. You don't have to read files into memory at once , Temporary memory is small .
    2. If the file is large , Read the entire file at once , It can take a long time . Use stream , You can read the file bit by bit , Read a little and you can go back to response, Faster response time .
    3. Node.js You can use pipes to transfer data between readable and writable streams , It's also easy to use .

Reference material :

koa-static file :https://github.com/koajs/static

koa-static Source code :https://github.com/koajs/static/blob/master/index.js

koa-send file :https://github.com/koajs/send

koa-send Source code :https://github.com/koajs/send/blob/master/index.js

At the end of the article , Thank you for your precious time reading this article , If this article gives you a little help or inspiration , Please don't be stingy with your praise and GitHub Little star , Your support is the motivation for the author to continue to create .

Author's blog GitHub Project address : https://github.com/dennis-jiang/Front-End-Knowledges

I also made a official account [ The big front of the attack ], No advertising , Don't write about hydrology , Only high quality original , Welcome to your attention ~

版权声明
本文为[A kind of Jiang Pengfei]所创,转载请带上原文链接,感谢
https://chowdera.com/2020/12/20201207123635966v.html