How do you create a gzip njson file in memory for ...
# google-cloud
i
How do you create a gzip njson file in memory for BucketObject in GCS?
Copy code
var jsonAsset = new pulumi.asset.StringAsset(`{"msg":"hello world"}\n`);

    this.jsonGZipFile = new gcp.storage.BucketObject(
      'myfolder/json-template.log.gz',
      {
        name: 'myfolder/json-template.log.gz',
        bucket: pulumi.interpolate`${args.bucket}`,
        contentType: 'application/x-ndjson',
        contentEncoding: 'gzip',
        source: jsonAsset,
      }
    );
File is created, but its an empty zip file.
Only way i could get it to work was to create the gzip file on disk and use the
source
property on the BuckObject. I had hoped to keep it in memory. I used these functions and used
fs.write
to create a valid ndjson gzip file.
Copy code
createNDJSONBuffer(data: any[]): Buffer {
    const ndjsonData = data.map(item => JSON.stringify(item)).join('\n');
    return Buffer.from(ndjsonData);
  }

  compressToGzipBuffer(inputBuffer: Buffer): Promise<Buffer> {
    return new Promise((resolve, reject) => {
      zlib.gzip(inputBuffer, (err, result) => {
        if (err) {
          reject(err);
        } else {
          resolve(result);
        }
      });
    });
  }