bytelyst/learning_ai_notes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
db7b9f8ff1
learning_ai_notes/backend/node_modules/js-tokens
History
saravanakumardb1 4fb859c81c feat(notes): scaffold product surfaces and backend MCP slices
2026-03-10 08:53:07 -07:00
..
index.d.ts feat(notes): scaffold product surfaces and backend MCP slices 2026-03-10 08:53:07 -07:00
index.js feat(notes): scaffold product surfaces and backend MCP slices 2026-03-10 08:53:07 -07:00
LICENSE feat(notes): scaffold product surfaces and backend MCP slices 2026-03-10 08:53:07 -07:00
package.json feat(notes): scaffold product surfaces and backend MCP slices 2026-03-10 08:53:07 -07:00
README.md feat(notes): scaffold product surfaces and backend MCP slices 2026-03-10 08:53:07 -07:00

README.md

js-tokens

The tiny, regex powered, lenient, almost spec-compliant JavaScript tokenizer that never fails.

const jsTokens = require("js-tokens");

const jsString = 'JSON.stringify({k:3.14**2}, null /*replacer*/, "\\t")';

Array.from(jsTokens(jsString), (token) => token.value).join("|");
// JSON|.|stringify|(|{|k|:|3.14|**|2|}|,| |null| |/*replacer*/|,| |"\t"|)

➡️ Full readme