Welcome to the Linux Foundation Forum!

opendirSync vs opendir

Hello everyone. In ch-13, in the streaming directory content example, opendirSync(__dirname) is used as input for Readable.from() method. I wonder why it was used instead of the async opendir method? As I understand it's not recommended to use sync methods when serving HTTP responses. Was it just for demonstrative purposes? Maybe I missed something? Thanks

'use strict'
const { createServer } = require('http')
const { Readable, Transform, pipeline } = require('stream')
const { opendirSync } = require('fs')

const createEntryStream = () => {
let syntax = '[\n'
return new Transform({
writableObjectMode: true,
readableObjectMode: false,
transform (entry, enc, next) {
next(null, ${syntax} "${entry.name}")
syntax = ',\n'
},
final (cb) {
this.push('\n]\n')
cb()
}
})
}

createServer((req, res) => {
if (req.url !== '/') {
res.statusCode = 404
res.end('Not Found')
return
}
const dirStream = Readable.from(opendirSync(__dirname))
const entryStream = createEntryStream()
res.setHeader('Content-Type', 'application/json')
pipeline(dirStream, entryStream, res, (err) => {
if (err) console.error(err)
})
}).listen(3000)

Comments

  • hi @mmd391

    As I understand it's not recommended to use sync methods when serving HTTP responses.

    This is correct, and we are not using sync methods when serving HTTP responses. We're using a sync method on initialization - which is what sync methods are for - convenient init stuff.

    It returns a stream (which is an asynchronous abstraction) which is then used during the request/response phase.

  • Hello @davidmarkclements

    thank you for your reply! So, if I would use const dirStream = Readable.from(await opendir(__dirname)), I would not gain much and only increase the complexity of the code? Sorry, if it's a silly question.

  • @mmd391 actually I have this wrong, I misread it. The sync op is in a request hot path - you're absolutely right

    I'm updated the code to the following:

    'use strict'
    const { createServer } = require('http')
    const { Readable, Transform, pipeline } = require('stream')
    const { opendir } = require('fs')
    
    const createEntryStream = () => {
      let syntax = '[\n'
      return new Transform({
        writableObjectMode: true,
        readableObjectMode: false,
        transform (entry, enc, next) {
          next(null, `${syntax} "${entry.name}"`)
          syntax = ',\n'
        },
        final (cb) {
          this.push('\n]\n')
          cb()
        }
      })
    }
    
    createServer((req, res) => {
      if (req.url !== '/') {
        res.statusCode = 404
        res.end('Not Found')
        return
      }
      opendir(__dirname, (err, dir) => {
        if (err) {
          res.statusCode = 500
          res.end('Server Error')
          return
        }
        const dirStream = Readable.from(dir)
        const entryStream = createEntryStream()
        res.setHeader('Content-Type', 'application/json')
        pipeline(dirStream, entryStream, res, (err) => {
          if (err) console.error(err)
        })
      })
    }).listen(3000)
    

    You could also use the following:

    'use strict'
    const { createServer } = require('http')
    const { Readable, Transform, pipeline } = require('stream')
    const { opendir } = require('fs').promises
    
    const createEntryStream = () => {
      let syntax = '[\n'
      return new Transform({
        writableObjectMode: true,
        readableObjectMode: false,
        transform (entry, enc, next) {
          next(null, `${syntax} "${entry.name}"`)
          syntax = ',\n'
        },
        final (cb) {
          this.push('\n]\n')
          cb()
        }
      })
    }
    
    
    createServer(async (req, res) => {
      if (req.url !== '/') {
      res.statusCode = 404
      res.end('Not Found')
      return
      }
      try { 
        const dirStream = Readable.from(await opendir(__dirname))
        const entryStream = createEntryStream()
        res.setHeader('Content-Type', 'application/json')
        pipeline(dirStream, entryStream, res, (err) => {
          if (err) console.error(err)
        })
      } catch (err) {
        res.statusCode = 500
        res.end('Server Error')
        return
      }
    }).listen(3000)
    

    Thanks for calling this out!

  • Thank you @davidmarkclements!

This discussion has been closed.