This message was deleted.

Only way i could get it to work was to create the gzip file on disk and use the

source

property on the BuckObject. I had hoped to keep it in memory. I used these functions and used

fs.write

to create a valid ndjson gzip file.

createNDJSONBuffer(data: any[]): Buffer {
    const ndjsonData = data.map(item => JSON.stringify(item)).join('\n');
    return Buffer.from(ndjsonData);
  }

  compressToGzipBuffer(inputBuffer: Buffer): Promise<Buffer> {
    return new Promise((resolve, reject) => {
      zlib.gzip(inputBuffer, (err, result) => {
        if (err) {
          reject(err);
        } else {
          resolve(result);
        }
      });
    });
  }