Brunobkr/llama.cpp_AlgMor24_github
ΩFFFΣLLIa • llama.cpp • AlgMor24 ██████╗ ███████╗███████╗███████╗██╗ ██╗ ██╗ █████╗ ██╔═══██╗██╔════╝██╔════╝██╔════╝██║ ██║ ██║██╔══██╗ ██║ ██║█████╗ █████╗ █████╗ ██║ ██║ ██║███████║ ██║ ██║██╔══╝ ██╔══╝ ██╔══╝ ██║ ██║ ██║██╔══██║ ╚██████╔╝██║ ██║ ███████╗███████╗███████╗██║██║ ██║ ╚═════╝ ╚═╝ ╚═╝ ╚══════╝╚══════╝╚══════╝╚═╝╚═╝ ╚═╝ High-Performance LLM / VLM Inference & Autonomous Agentic Ecosystem… See the full description on the dataset page: https://huggingface.co/datasets/Brunobkr/llama.cpp_AlgMor24_github.
03.1k
1# minipass2 3A _very_ minimal implementation of a [PassThrough4stream](https://nodejs.org/api/stream.html#stream_class_stream_passthrough)5 6[It's very7fast](https://docs.google.com/spreadsheets/d/1K_HR5oh3r80b8WVMWCPPjfuWXUgfkmhlX7FGI6JJ8tY/edit?usp=sharing)8for objects, strings, and buffers.9 10Supports `pipe()`ing (including multi-`pipe()` and backpressure11transmission), buffering data until either a `data` event handler12or `pipe()` is added (so you don't lose the first chunk), and13most other cases where PassThrough is a good idea.14 15There is a `read()` method, but it's much more efficient to16consume data from this stream via `'data'` events or by calling17`pipe()` into some other stream. Calling `read()` requires the18buffer to be flattened in some cases, which requires copying19memory.20 21If you set `objectMode: true` in the options, then whatever is22written will be emitted. Otherwise, it'll do a minimal amount of23Buffer copying to ensure proper Streams semantics when `read(n)`24is called.25 26`objectMode` can only be set at instantiation. Attempting to27write something other than a String or Buffer without having set28`objectMode` in the options will throw an error.29 30This is not a `through` or `through2` stream. It doesn't31transform the data, it just passes it right through. If you want32to transform the data, extend the class, and override the33`write()` method. Once you're done transforming the data however34you want, call `super.write()` with the transform output.35 36For some examples of streams that extend Minipass in various37ways, check out:38 39- [minizlib](http://npm.im/minizlib)40- [fs-minipass](http://npm.im/fs-minipass)41- [tar](http://npm.im/tar)42- [minipass-collect](http://npm.im/minipass-collect)43- [minipass-flush](http://npm.im/minipass-flush)44- [minipass-pipeline](http://npm.im/minipass-pipeline)45- [tap](http://npm.im/tap)46- [tap-parser](http://npm.im/tap-parser)47- [treport](http://npm.im/treport)48- [minipass-fetch](http://npm.im/minipass-fetch)49- [pacote](http://npm.im/pacote)50- [make-fetch-happen](http://npm.im/make-fetch-happen)51- [cacache](http://npm.im/cacache)52- [ssri](http://npm.im/ssri)53- [npm-registry-fetch](http://npm.im/npm-registry-fetch)54- [minipass-json-stream](http://npm.im/minipass-json-stream)55- [minipass-sized](http://npm.im/minipass-sized)56 57## Usage in TypeScript58 59The `Minipass` class takes three type template definitions:60 61- `RType` the type being read, which defaults to `Buffer`. If62 `RType` is `string`, then the constructor _must_ get an options63 object specifying either an `encoding` or `objectMode: true`.64 If it's anything other than `string` or `Buffer`, then it65 _must_ get an options object specifying `objectMode: true`.66- `WType` the type being written. If `RType` is `Buffer` or67 `string`, then this defaults to `ContiguousData` (Buffer,68 string, ArrayBuffer, or ArrayBufferView). Otherwise, it69 defaults to `RType`.70- `Events` type mapping event names to the arguments emitted71 with that event, which extends `Minipass.Events`.72 73To declare types for custom events in subclasses, extend the74third parameter with your own event signatures. For example:75 76```js77import { Minipass } from 'minipass'78 79// a NDJSON stream that emits 'jsonError' when it can't stringify80export interface Events extends Minipass.Events {81 jsonError: [e: Error]82}83 84export class NDJSONStream extends Minipass<string, any, Events> {85 constructor() {86 super({ objectMode: true })87 }88 89 // data is type `any` because that's WType90 write(data, encoding, cb) {91 try {92 const json = JSON.stringify(data)93 return super.write(json + '\n', encoding, cb)94 } catch (er) {95 if (!er instanceof Error) {96 er = Object.assign(new Error('json stringify failed'), {97 cause: er,98 })99 }100 // trying to emit with something OTHER than an error will101 // fail, because we declared the event arguments type.102 this.emit('jsonError', er)103 }104 }105}106 107const s = new NDJSONStream()108s.on('jsonError', e => {109 // here, TS knows that e is an Error110})111```112 113Emitting/handling events that aren't declared in this way is114fine, but the arguments will be typed as `unknown`.115 116## Differences from Node.js Streams117 118There are several things that make Minipass streams different119from (and in some ways superior to) Node.js core streams.120 121Please read these caveats if you are familiar with node-core122streams and intend to use Minipass streams in your programs.123 124You can avoid most of these differences entirely (for a very125small performance penalty) by setting `{async: true}` in the126constructor options.127 128### Timing129 130Minipass streams are designed to support synchronous use-cases.131Thus, data is emitted as soon as it is available, always. It is132buffered until read, but no longer. Another way to look at it is133that Minipass streams are exactly as synchronous as the logic134that writes into them.135 136This can be surprising if your code relies on137`PassThrough.write()` always providing data on the next tick138rather than the current one, or being able to call `resume()` and139not have the entire buffer disappear immediately.140 141However, without this synchronicity guarantee, there would be no142way for Minipass to achieve the speeds it does, or support the143synchronous use cases that it does. Simply put, waiting takes144time.145 146This non-deferring approach makes Minipass streams much easier to147reason about, especially in the context of Promises and other148flow-control mechanisms.149 150Example:151 152```js153// hybrid module, either works154import { Minipass } from 'minipass'155// or:156const { Minipass } = require('minipass')157 158const stream = new Minipass()159stream.on('data', () => console.log('data event'))160console.log('before write')161stream.write('hello')162console.log('after write')163// output:164// before write165// data event166// after write167```168 169### Exception: Async Opt-In170 171If you wish to have a Minipass stream with behavior that more172closely mimics Node.js core streams, you can set the stream in173async mode either by setting `async: true` in the constructor174options, or by setting `stream.async = true` later on.175 176```js177// hybrid module, either works178import { Minipass } from 'minipass'179// or:180const { Minipass } = require('minipass')181 182const asyncStream = new Minipass({ async: true })183asyncStream.on('data', () => console.log('data event'))184console.log('before write')185asyncStream.write('hello')186console.log('after write')187// output:188// before write189// after write190// data event <-- this is deferred until the next tick191```192 193Switching _out_ of async mode is unsafe, as it could cause data194corruption, and so is not enabled. Example:195 196```js197import { Minipass } from 'minipass'198const stream = new Minipass({ encoding: 'utf8' })199stream.on('data', chunk => console.log(chunk))200stream.async = true201console.log('before writes')202stream.write('hello')203setStreamSyncAgainSomehow(stream) // <-- this doesn't actually exist!204stream.write('world')205console.log('after writes')206// hypothetical output would be:207// before writes208// world209// after writes210// hello211// NOT GOOD!212```213 214To avoid this problem, once set into async mode, any attempt to215make the stream sync again will be ignored.216 217```js218const { Minipass } = require('minipass')219const stream = new Minipass({ encoding: 'utf8' })220stream.on('data', chunk => console.log(chunk))221stream.async = true222console.log('before writes')223stream.write('hello')224stream.async = false // <-- no-op, stream already async225stream.write('world')226console.log('after writes')227// actual output:228// before writes229// after writes230// hello231// world232```233 234### No High/Low Water Marks235 236Node.js core streams will optimistically fill up a buffer,237returning `true` on all writes until the limit is hit, even if238the data has nowhere to go. Then, they will not attempt to draw239more data in until the buffer size dips below a minimum value.240 241Minipass streams are much simpler. The `write()` method will242return `true` if the data has somewhere to go (which is to say,243given the timing guarantees, that the data is already there by244the time `write()` returns).245 246If the data has nowhere to go, then `write()` returns false, and247the data sits in a buffer, to be drained out immediately as soon248as anyone consumes it.249 250Since nothing is ever buffered unnecessarily, there is much less251copying data, and less bookkeeping about buffer capacity levels.252 253### Hazards of Buffering (or: Why Minipass Is So Fast)254 255Since data written to a Minipass stream is immediately written256all the way through the pipeline, and `write()` always returns257true/false based on whether the data was fully flushed,258backpressure is communicated immediately to the upstream caller.259This minimizes buffering.260 261Consider this case:262 263```js264const { PassThrough } = require('stream')265const p1 = new PassThrough({ highWaterMark: 1024 })266const p2 = new PassThrough({ highWaterMark: 1024 })267const p3 = new PassThrough({ highWaterMark: 1024 })268const p4 = new PassThrough({ highWaterMark: 1024 })269 270p1.pipe(p2).pipe(p3).pipe(p4)271p4.on('data', () => console.log('made it through'))272 273// this returns false and buffers, then writes to p2 on next tick (1)274// p2 returns false and buffers, pausing p1, then writes to p3 on next tick (2)275// p3 returns false and buffers, pausing p2, then writes to p4 on next tick (3)276// p4 returns false and buffers, pausing p3, then emits 'data' and 'drain'277// on next tick (4)278// p3 sees p4's 'drain' event, and calls resume(), emitting 'resume' and279// 'drain' on next tick (5)280// p2 sees p3's 'drain', calls resume(), emits 'resume' and 'drain' on next tick (6)281// p1 sees p2's 'drain', calls resume(), emits 'resume' and 'drain' on next282// tick (7)283 284p1.write(Buffer.alloc(2048)) // returns false285```286 287Along the way, the data was buffered and deferred at each stage,288and multiple event deferrals happened, for an unblocked pipeline289where it was perfectly safe to write all the way through!290 291Furthermore, setting a `highWaterMark` of `1024` might lead292someone reading the code to think an advisory maximum of 1KiB is293being set for the pipeline. However, the actual advisory294buffering level is the _sum_ of `highWaterMark` values, since295each one has its own bucket.296 297Consider the Minipass case:298 299```js300const m1 = new Minipass()301const m2 = new Minipass()302const m3 = new Minipass()303const m4 = new Minipass()304 305m1.pipe(m2).pipe(m3).pipe(m4)306m4.on('data', () => console.log('made it through'))307 308// m1 is flowing, so it writes the data to m2 immediately309// m2 is flowing, so it writes the data to m3 immediately310// m3 is flowing, so it writes the data to m4 immediately311// m4 is flowing, so it fires the 'data' event immediately, returns true312// m4's write returned true, so m3 is still flowing, returns true313// m3's write returned true, so m2 is still flowing, returns true314// m2's write returned true, so m1 is still flowing, returns true315// No event deferrals or buffering along the way!316 317m1.write(Buffer.alloc(2048)) // returns true318```319 320It is extremely unlikely that you _don't_ want to buffer any data321written, or _ever_ buffer data that can be flushed all the way322through. Neither node-core streams nor Minipass ever fail to323buffer written data, but node-core streams do a lot of324unnecessary buffering and pausing.325 326As always, the faster implementation is the one that does less327stuff and waits less time to do it.328 329### Immediately emit `end` for empty streams (when not paused)330 331If a stream is not paused, and `end()` is called before writing332any data into it, then it will emit `end` immediately.333 334If you have logic that occurs on the `end` event which you don't335want to potentially happen immediately (for example, closing file336descriptors, moving on to the next entry in an archive parse337stream, etc.) then be sure to call `stream.pause()` on creation,338and then `stream.resume()` once you are ready to respond to the339`end` event.340 341However, this is _usually_ not a problem because:342 343### Emit `end` When Asked344 345One hazard of immediately emitting `'end'` is that you may not346yet have had a chance to add a listener. In order to avoid this347hazard, Minipass streams safely re-emit the `'end'` event if a348new listener is added after `'end'` has been emitted.349 350Ie, if you do `stream.on('end', someFunction)`, and the stream351has already emitted `end`, then it will call the handler right352away. (You can think of this somewhat like attaching a new353`.then(fn)` to a previously-resolved Promise.)354 355To prevent calling handlers multiple times who would not expect356multiple ends to occur, all listeners are removed from the357`'end'` event whenever it is emitted.358 359### Emit `error` When Asked360 361The most recent error object passed to the `'error'` event is362stored on the stream. If a new `'error'` event handler is added,363and an error was previously emitted, then the event handler will364be called immediately (or on `process.nextTick` in the case of365async streams).366 367This makes it much more difficult to end up trying to interact368with a broken stream, if the error handler is added after an369error was previously emitted.370 371### Impact of "immediate flow" on Tee-streams372 373A "tee stream" is a stream piping to multiple destinations:374 375```js376const tee = new Minipass()377t.pipe(dest1)378t.pipe(dest2)379t.write('foo') // goes to both destinations380```381 382Since Minipass streams _immediately_ process any pending data383through the pipeline when a new pipe destination is added, this384can have surprising effects, especially when a stream comes in385from some other function and may or may not have data in its386buffer.387 388```js389// WARNING! WILL LOSE DATA!390const src = new Minipass()391src.write('foo')392src.pipe(dest1) // 'foo' chunk flows to dest1 immediately, and is gone393src.pipe(dest2) // gets nothing!394```395 396One solution is to create a dedicated tee-stream junction that397pipes to both locations, and then pipe to _that_ instead.398 399```js400// Safe example: tee to both places401const src = new Minipass()402src.write('foo')403const tee = new Minipass()404tee.pipe(dest1)405tee.pipe(dest2)406src.pipe(tee) // tee gets 'foo', pipes to both locations407```408 409The same caveat applies to `on('data')` event listeners. The410first one added will _immediately_ receive all of the data,411leaving nothing for the second:412 413```js414// WARNING! WILL LOSE DATA!415const src = new Minipass()416src.write('foo')417src.on('data', handler1) // receives 'foo' right away418src.on('data', handler2) // nothing to see here!419```420 421Using a dedicated tee-stream can be used in this case as well:422 423```js424// Safe example: tee to both data handlers425const src = new Minipass()426src.write('foo')427const tee = new Minipass()428tee.on('data', handler1)429tee.on('data', handler2)430src.pipe(tee)431```432 433All of the hazards in this section are avoided by setting `{434async: true }` in the Minipass constructor, or by setting435`stream.async = true` afterwards. Note that this does add some436overhead, so should only be done in cases where you are willing437to lose a bit of performance in order to avoid having to refactor438program logic.439 440## USAGE441 442It's a stream! Use it like a stream and it'll most likely do what443you want.444 445```js446import { Minipass } from 'minipass'447const mp = new Minipass(options) // options is optional448mp.write('foo')449mp.pipe(someOtherStream)450mp.end('bar')451```452 453### OPTIONS454 455- `encoding` How would you like the data coming _out_ of the456 stream to be encoded? Accepts any values that can be passed to457 `Buffer.toString()`.458- `objectMode` Emit data exactly as it comes in. This will be459 flipped on by default if you write() something other than a460 string or Buffer at any point. Setting `objectMode: true` will461 prevent setting any encoding value.462- `async` Defaults to `false`. Set to `true` to defer data463 emission until next tick. This reduces performance slightly,464 but makes Minipass streams use timing behavior closer to Node465 core streams. See [Timing](#timing) for more details.466- `signal` An `AbortSignal` that will cause the stream to unhook467 itself from everything and become as inert as possible. Note468 that providing a `signal` parameter will make `'error'` events469 no longer throw if they are unhandled, but they will still be470 emitted to handlers if any are attached.471 472### API473 474Implements the user-facing portions of Node.js's `Readable` and475`Writable` streams.476 477### Methods478 479- `write(chunk, [encoding], [callback])` - Put data in. (Note480 that, in the base Minipass class, the same data will come out.)481 Returns `false` if the stream will buffer the next write, or482 true if it's still in "flowing" mode.483- `end([chunk, [encoding]], [callback])` - Signal that you have484 no more data to write. This will queue an `end` event to be485 fired when all the data has been consumed.486- `pause()` - No more data for a while, please. This also487 prevents `end` from being emitted for empty streams until the488 stream is resumed.489- `resume()` - Resume the stream. If there's data in the buffer,490 it is all discarded. Any buffered events are immediately491 emitted.492- `pipe(dest)` - Send all output to the stream provided. When493 data is emitted, it is immediately written to any and all pipe494 destinations. (Or written on next tick in `async` mode.)495- `unpipe(dest)` - Stop piping to the destination stream. This is496 immediate, meaning that any asynchronously queued data will497 _not_ make it to the destination when running in `async` mode.498 - `options.end` - Boolean, end the destination stream when the499 source stream ends. Default `true`.500 - `options.proxyErrors` - Boolean, proxy `error` events from501 the source stream to the destination stream. Note that errors502 are _not_ proxied after the pipeline terminates, either due503 to the source emitting `'end'` or manually unpiping with504 `src.unpipe(dest)`. Default `false`.505- `on(ev, fn)`, `emit(ev, fn)` - Minipass streams are506 EventEmitters. Some events are given special treatment,507 however. (See below under "events".)508- `promise()` - Returns a Promise that resolves when the stream509 emits `end`, or rejects if the stream emits `error`.510- `collect()` - Return a Promise that resolves on `end` with an511 array containing each chunk of data that was emitted, or512 rejects if the stream emits `error`. Note that this consumes513 the stream data.514- `concat()` - Same as `collect()`, but concatenates the data515 into a single Buffer object. Will reject the returned promise516 if the stream is in objectMode, or if it goes into objectMode517 by the end of the data.518- `read(n)` - Consume `n` bytes of data out of the buffer. If `n`519 is not provided, then consume all of it. If `n` bytes are not520 available, then it returns null. **Note** consuming streams in521 this way is less efficient, and can lead to unnecessary Buffer522 copying.523- `destroy([er])` - Destroy the stream. If an error is provided,524 then an `'error'` event is emitted. If the stream has a525 `close()` method, and has not emitted a `'close'` event yet,526 then `stream.close()` will be called. Any Promises returned by527 `.promise()`, `.collect()` or `.concat()` will be rejected.528 After being destroyed, writing to the stream will emit an529 error. No more data will be emitted if the stream is destroyed,530 even if it was previously buffered.531 532### Properties533 534- `bufferLength` Read-only. Total number of bytes buffered, or in535 the case of objectMode, the total number of objects.536- `encoding` Read-only. The encoding that has been set.537- `flowing` Read-only. Boolean indicating whether a chunk written538 to the stream will be immediately emitted.539- `emittedEnd` Read-only. Boolean indicating whether the end-ish540 events (ie, `end`, `prefinish`, `finish`) have been emitted.541 Note that listening on any end-ish event will immediateyl542 re-emit it if it has already been emitted.543- `writable` Whether the stream is writable. Default `true`. Set544 to `false` when `end()`545- `readable` Whether the stream is readable. Default `true`.546- `pipes` An array of Pipe objects referencing streams that this547 stream is piping into.548- `destroyed` A getter that indicates whether the stream was549 destroyed.550- `paused` True if the stream has been explicitly paused,551 otherwise false.552- `objectMode` Indicates whether the stream is in `objectMode`.553- `aborted` Readonly property set when the `AbortSignal`554 dispatches an `abort` event.555 556### Events557 558- `data` Emitted when there's data to read. Argument is the data559 to read. This is never emitted while not flowing. If a listener560 is attached, that will resume the stream.561- `end` Emitted when there's no more data to read. This will be562 emitted immediately for empty streams when `end()` is called.563 If a listener is attached, and `end` was already emitted, then564 it will be emitted again. All listeners are removed when `end`565 is emitted.566- `prefinish` An end-ish event that follows the same logic as567 `end` and is emitted in the same conditions where `end` is568 emitted. Emitted after `'end'`.569- `finish` An end-ish event that follows the same logic as `end`570 and is emitted in the same conditions where `end` is emitted.571 Emitted after `'prefinish'`.572- `close` An indication that an underlying resource has been573 released. Minipass does not emit this event, but will defer it574 until after `end` has been emitted, since it throws off some575 stream libraries otherwise.576- `drain` Emitted when the internal buffer empties, and it is577 again suitable to `write()` into the stream.578- `readable` Emitted when data is buffered and ready to be read579 by a consumer.580- `resume` Emitted when stream changes state from buffering to581 flowing mode. (Ie, when `resume` is called, `pipe` is called,582 or a `data` event listener is added.)583 584### Static Methods585 586- `Minipass.isStream(stream)` Returns `true` if the argument is a587 stream, and false otherwise. To be considered a stream, the588 object must be either an instance of Minipass, or an589 EventEmitter that has either a `pipe()` method, or both590 `write()` and `end()` methods. (Pretty much any stream in591 node-land will return `true` for this.)592 593## EXAMPLES594 595Here are some examples of things you can do with Minipass596streams.597 598### simple "are you done yet" promise599 600```js601mp.promise().then(602 () => {603 // stream is finished604 },605 er => {606 // stream emitted an error607 }608)609```610 611### collecting612 613```js614mp.collect().then(all => {615 // all is an array of all the data emitted616 // encoding is supported in this case, so617 // so the result will be a collection of strings if618 // an encoding is specified, or buffers/objects if not.619 //620 // In an async function, you may do621 // const data = await stream.collect()622})623```624 625### collecting into a single blob626 627This is a bit slower because it concatenates the data into one628chunk for you, but if you're going to do it yourself anyway, it's629convenient this way:630 631```js632mp.concat().then(onebigchunk => {633 // onebigchunk is a string if the stream634 // had an encoding set, or a buffer otherwise.635})636```637 638### iteration639 640You can iterate over streams synchronously or asynchronously in641platforms that support it.642 643Synchronous iteration will end when the currently available data644is consumed, even if the `end` event has not been reached. In645string and buffer mode, the data is concatenated, so unless646multiple writes are occurring in the same tick as the `read()`,647sync iteration loops will generally only have a single iteration.648 649To consume chunks in this way exactly as they have been written,650with no flattening, create the stream with the `{ objectMode:651true }` option.652 653```js654const mp = new Minipass({ objectMode: true })655mp.write('a')656mp.write('b')657for (let letter of mp) {658 console.log(letter) // a, b659}660mp.write('c')661mp.write('d')662for (let letter of mp) {663 console.log(letter) // c, d664}665mp.write('e')666mp.end()667for (let letter of mp) {668 console.log(letter) // e669}670for (let letter of mp) {671 console.log(letter) // nothing672}673```674 675Asynchronous iteration will continue until the end event is reached,676consuming all of the data.677 678```js679const mp = new Minipass({ encoding: 'utf8' })680 681// some source of some data682let i = 5683const inter = setInterval(() => {684 if (i-- > 0) mp.write(Buffer.from('foo\n', 'utf8'))685 else {686 mp.end()687 clearInterval(inter)688 }689}, 100)690 691// consume the data with asynchronous iteration692async function consume() {693 for await (let chunk of mp) {694 console.log(chunk)695 }696 return 'ok'697}698 699consume().then(res => console.log(res))700// logs `foo\n` 5 times, and then `ok`701```702 703### subclass that `console.log()`s everything written into it704 705```js706class Logger extends Minipass {707 write(chunk, encoding, callback) {708 console.log('WRITE', chunk, encoding)709 return super.write(chunk, encoding, callback)710 }711 end(chunk, encoding, callback) {712 console.log('END', chunk, encoding)713 return super.end(chunk, encoding, callback)714 }715}716 717someSource.pipe(new Logger()).pipe(someDest)718```719 720### same thing, but using an inline anonymous class721 722```js723// js classes are fun724someSource725 .pipe(726 new (class extends Minipass {727 emit(ev, ...data) {728 // let's also log events, because debugging some weird thing729 console.log('EMIT', ev)730 return super.emit(ev, ...data)731 }732 write(chunk, encoding, callback) {733 console.log('WRITE', chunk, encoding)734 return super.write(chunk, encoding, callback)735 }736 end(chunk, encoding, callback) {737 console.log('END', chunk, encoding)738 return super.end(chunk, encoding, callback)739 }740 })()741 )742 .pipe(someDest)743```744 745### subclass that defers 'end' for some reason746 747```js748class SlowEnd extends Minipass {749 emit(ev, ...args) {750 if (ev === 'end') {751 console.log('going to end, hold on a sec')752 setTimeout(() => {753 console.log('ok, ready to end now')754 super.emit('end', ...args)755 }, 100)756 return true757 } else {758 return super.emit(ev, ...args)759 }760 }761}762```763 764### transform that creates newline-delimited JSON765 766```js767class NDJSONEncode extends Minipass {768 write(obj, cb) {769 try {770 // JSON.stringify can throw, emit an error on that771 return super.write(JSON.stringify(obj) + '\n', 'utf8', cb)772 } catch (er) {773 this.emit('error', er)774 }775 }776 end(obj, cb) {777 if (typeof obj === 'function') {778 cb = obj779 obj = undefined780 }781 if (obj !== undefined) {782 this.write(obj)783 }784 return super.end(cb)785 }786}787```788 789### transform that parses newline-delimited JSON790 791```js792class NDJSONDecode extends Minipass {793 constructor(options) {794 // always be in object mode, as far as Minipass is concerned795 super({ objectMode: true })796 this._jsonBuffer = ''797 }798 write(chunk, encoding, cb) {799 if (800 typeof chunk === 'string' &&801 typeof encoding === 'string' &&802 encoding !== 'utf8'803 ) {804 chunk = Buffer.from(chunk, encoding).toString()805 } else if (Buffer.isBuffer(chunk)) {806 chunk = chunk.toString()807 }808 if (typeof encoding === 'function') {809 cb = encoding810 }811 const jsonData = (this._jsonBuffer + chunk).split('\n')812 this._jsonBuffer = jsonData.pop()813 for (let i = 0; i < jsonData.length; i++) {814 try {815 // JSON.parse can throw, emit an error on that816 super.write(JSON.parse(jsonData[i]))817 } catch (er) {818 this.emit('error', er)819 continue820 }821 }822 if (cb) cb()823 }824}825```826 