Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Thanks to this post I learned about Lark, which looks like a really nice parser library for Python.

The JSON tutorial on their site is excellent - shows how to build a basic parser for JSON, then goes into some great detail about how to improve its performance: https://lark-parser.readthedocs.io/en/latest/json_tutorial.h...

Here's the grammar used for the RDBMS project: https://github.com/spandanb/learndb-py/blob/master/learndb/l...



Highly recommend Lark for Python projects -- it's easy to use :)

Their IDE was super useful for debugging the grammar: https://www.lark-parser.org/ide/

We use Lark for a SQL-like language tailored for using AI models in EvaDB: https://github.com/georgia-tech-db/evadb/blob/master/evadb/p... https://github.com/georgia-tech-db/evadb/

If you like Lark, please consider sponsoring them: https://github.com/sponsors/lark-parser


DSL in a string? Is that 'really nice'? I haven't used or needed this in Python that I can think of, but surely we can do better than that?

Even a dict with expected keys and construction via the bitwise or operator (which would roughly match the form of a lot of the grammar) would be better wouldn't it? Imports could be imports, just mixed in somehow.

This is just first thoughts at a glance, maybe I'm missing something.


Lark supports, and recommends, writing and storing the grammar in a .lark file. We have syntax highlighting support in all major IDEs, and even in github itself. For example, here is Lark's built-in grammar for Python: https://github.com/lark-parser/lark/blob/master/lark/grammar...

The rationale is that it's more terse and has less visual clutter than a DSL over Python, which makes it easier to read and write.


It’s an existing, standard, language for describing grammars. Quite clearly the trade-off being made here. Seemingly not the one you would’ve made. Don’t act like it’s objectively bad.


in a string. I didn't say the DSL was novel. I learnt BNF too. (It does have extras that are either unique to it or standard beyond my familiarity though.)


I was always surprised at how the Ruby community talks about DSLs when using Ruby classes and catch all methods, overloading, etc. Nothing bad with it, but I feel it is not really a language, just very dynamic Ruby code.

A DSL that is a string, while having some downsides, does not have the arbitrary limitations of the "host" language. Either approach has its pros an cons.


I'm not that versed in Ruby, but I think similarly say the Django ORM in Python borders on DSL - especially with use of bitwise operators to build a query.

It's the tooling aspect that's my gripe with it in a string really, it makes it less likely (and more editor-specific) that I can have syntax highlighting, LSP, etc. That's incidentally the only way in which I don't prefer SQL to Django ORM - i.e. it really isn't that it's a DSL (SQL) that bothers me, it's the string.

But hopefully obviously it's not a big deal, I was just surprised at the look of it when I saw it described as 'really nice' and that the project describes itself as 'focussed on ergonomics'. It just doesn't seem brilliant to me, fine perhaps, par for the course apparently, but not remarkable.


That's pretty standard for parsing libraries - have you seen any good ones (for Python or other languages) that don't use a DSL like this?

The only one I've seen is this one: https://parsy.readthedocs.io/en/latest/tutorial.html


Parse, comparse, parsec and PEGs come to mind.

1: https://github.com/massung/parse

2: https://wiki.call-cc.org/eggref/5/comparse

3: https://hackage.haskell.org/package/parsec

4: https://janet-lang.org/docs/peg.html


Here is another PEG one, though Guile allows expressing things as strings and as macro calls / s-expressions:

5: https://www.gnu.org/software/guile/manual/html_node/PEG-Pars...


̶T̶h̶i̶s̶ ̶s̶t̶r̶i̶n̶g̶ ̶i̶s̶ ̶t̶h̶e̶ ̶g̶r̶a̶m̶m̶a̶r̶ ̶t̶h̶a̶t̶ ̶l̶a̶r̶k̶ ̶u̶s̶e̶s̶ ̶t̶o̶ ̶p̶a̶r̶s̶e̶ ̶t̶h̶e̶ ̶u̶s̶e̶r̶ ̶s̶u̶b̶m̶i̶t̶t̶e̶d̶ ̶s̶q̶l̶ ̶i̶n̶t̶o̶ ̶a̶n̶ ̶A̶S̶T̶.̶ ̶T̶h̶i̶s̶ ̶p̶a̶r̶s̶i̶n̶g̶ ̶i̶s̶ ̶f̶a̶r̶ ̶m̶o̶r̶e̶ ̶c̶o̶m̶p̶l̶e̶t̶e̶ ̶a̶n̶d̶ ̶r̶o̶b̶u̶s̶t̶ ̶t̶h̶a̶n̶ ̶w̶h̶a̶t̶ ̶d̶o̶i̶n̶g̶ ̶t̶h̶i̶s̶ ̶i̶n̶ ̶p̶u̶r̶e̶ ̶p̶y̶t̶h̶o̶n̶ ̶w̶o̶u̶l̶d̶ ̶a̶l̶l̶o̶w̶ ̶(̶w̶i̶t̶h̶o̶u̶t̶ ̶o̶f̶ ̶c̶o̶u̶r̶s̶e̶ ̶i̶m̶p̶l̶e̶m̶e̶n̶t̶i̶n̶g̶ ̶t̶h̶e̶ ̶e̶n̶t̶i̶r̶e̶ ̶l̶e̶x̶e̶r̶ ̶a̶n̶d̶ ̶p̶a̶r̶s̶e̶r̶ ̶t̶h̶a̶t̶ ̶l̶a̶r̶k̶ ̶i̶m̶p̶l̶e̶m̶e̶n̶t̶s̶)̶.̶

T̶h̶e̶r̶e̶ ̶m̶a̶y̶ ̶b̶e̶ ̶s̶o̶m̶e̶ ̶p̶o̶s̶t̶ ̶p̶a̶r̶s̶i̶n̶g̶ ̶v̶a̶l̶i̶d̶a̶t̶i̶o̶n̶ ̶t̶h̶a̶t̶ ̶c̶a̶n̶ ̶b̶e̶ ̶d̶o̶n̶e̶ ̶h̶e̶r̶e̶-̶ ̶b̶u̶t̶ ̶t̶h̶a̶t̶ ̶w̶o̶u̶l̶d̶ ̶b̶e̶ ̶s̶o̶m̶e̶t̶h̶i̶n̶g̶ ̶t̶h̶a̶t̶'̶s̶ ̶b̶e̶y̶o̶n̶d̶ ̶t̶h̶e̶ ̶d̶o̶m̶a̶i̶n̶ ̶o̶f̶ ̶t̶h̶e̶ ̶p̶a̶r̶s̶e̶r̶.̶

Edit: I see what you mean. I surveyed a bunch a parser generator libraries, and they also seemed to use a text based DSL- rather than DSL based on python structures. What you're describing would have made the grammar development more ergonomic and simple.


It is https://en.m.wikipedia.org/wiki/Backus–Naur_form or something similar. A python DSL would certainly be less expressive and useful.


I realise that (with some additions, certainly - it has imports) I just don't think embedding it in a string is 'really nice', personally. The 'developer experience' will be crap or reliant on hyper-specific tooling; so likely crap. (The RHS of that 'or' will be too niche to be good and well-maintained, probably.)


Not to be coy or disrespectful( and this is excellent work and a way to learn new things) but if one is generating a parser as a means to an end(the end being execution of the AST on the database), what are the learnings one gets from just the parser bit? Is it some kind of optimization you need to keep doing to the generated parser to make it more efficient?

Would a logical next step be Generate an optimal query plan from the AST(somehow..)?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: