opendirSync vs opendir

mmd391 · November 2020

Hello everyone. In ch-13, in the streaming directory content example, opendirSync(__dirname) is used as input for Readable.from() method. I wonder why it was used instead of the async opendir method? As I understand it's not recommended to use sync methods when serving HTTP responses. Was it just for demonstrative purposes? Maybe I missed something? Thanks

'use strict'
const { createServer } = require('http')
const { Readable, Transform, pipeline } = require('stream')
const { opendirSync } = require('fs')

const createEntryStream = () => {
let syntax = '[\n'
return new Transform({
writableObjectMode: true,
readableObjectMode: false,
transform (entry, enc, next) {
next(null, ${syntax} "${entry.name}")
syntax = ',\n'
},
final (cb) {
this.push('\n]\n')
cb()
}
})
}

createServer((req, res) => {
if (req.url !== '/') {
res.statusCode = 404
res.end('Not Found')
return
}
const dirStream = Readable.from(opendirSync(__dirname))
const entryStream = createEntryStream()
res.setHeader('Content-Type', 'application/json')
pipeline(dirStream, entryStream, res, (err) => {
if (err) console.error(err)
})
}).listen(3000)

davidmarkclements · December 2020

hi @mmd391

As I understand it's not recommended to use sync methods when serving HTTP responses.

This is correct, and we are not using sync methods when serving HTTP responses. We're using a sync method on initialization - which is what sync methods are for - convenient init stuff.

It returns a stream (which is an asynchronous abstraction) which is then used during the request/response phase.

mmd391 · December 2020

Hello @davidmarkclements

thank you for your reply! So, if I would use const dirStream = Readable.from(await opendir(__dirname)), I would not gain much and only increase the complexity of the code? Sorry, if it's a silly question.

davidmarkclements · December 2020

@mmd391 actually I have this wrong, I misread it. The sync op is in a request hot path - you're absolutely right

I'm updated the code to the following:

'use strict'
const { createServer } = require('http')
const { Readable, Transform, pipeline } = require('stream')
const { opendir } = require('fs')

const createEntryStream = () => {
  let syntax = '[\n'
  return new Transform({
    writableObjectMode: true,
    readableObjectMode: false,
    transform (entry, enc, next) {
      next(null, `${syntax} "${entry.name}"`)
      syntax = ',\n'
    },
    final (cb) {
      this.push('\n]\n')
      cb()
    }
  })
}

createServer((req, res) => {
  if (req.url !== '/') {
    res.statusCode = 404
    res.end('Not Found')
    return
  }
  opendir(__dirname, (err, dir) => {
    if (err) {
      res.statusCode = 500
      res.end('Server Error')
      return
    }
    const dirStream = Readable.from(dir)
    const entryStream = createEntryStream()
    res.setHeader('Content-Type', 'application/json')
    pipeline(dirStream, entryStream, res, (err) => {
      if (err) console.error(err)
    })
  })
}).listen(3000)

You could also use the following:

'use strict'
const { createServer } = require('http')
const { Readable, Transform, pipeline } = require('stream')
const { opendir } = require('fs').promises

const createEntryStream = () => {
  let syntax = '[\n'
  return new Transform({
    writableObjectMode: true,
    readableObjectMode: false,
    transform (entry, enc, next) {
      next(null, `${syntax} "${entry.name}"`)
      syntax = ',\n'
    },
    final (cb) {
      this.push('\n]\n')
      cb()
    }
  })
}


createServer(async (req, res) => {
  if (req.url !== '/') {
  res.statusCode = 404
  res.end('Not Found')
  return
  }
  try { 
    const dirStream = Readable.from(await opendir(__dirname))
    const entryStream = createEntryStream()
    res.setHeader('Content-Type', 'application/json')
    pipeline(dirStream, entryStream, res, (err) => {
      if (err) console.error(err)
    })
  } catch (err) {
    res.statusCode = 500
    res.end('Server Error')
    return
  }
}).listen(3000)

Thanks for calling this out!

mmd391 · December 2020

Thank you @davidmarkclements!

opendirSync vs opendir

Comments