Docs for embedding Elvish?

[IngwiePhoenix]

What new feature should Elvish have?

I have long been wanting to build a build system to have an alternative to CMake - and as an excercise in graph building and processing of such. One very important part in just about any software build is the ability to use at least snippets of scripts - and I would like to use Elvish for that.

And thus, I'd like to know how to embed Elvish as a script language for my project - or more precisely, I would like to request this as a feature if it doesn't exist yet! :)

Output of "elvish -version"

0.21.0-dev.0.20240515223629-06979efb9a2a+official

Code of Conduct

• I agree to follow Elvish's Code of Conduct.

[krader1961]

Embedding any code written in Go inside an app written in another language is problematic. Some of the challenges are due to the Go runtime, how types are defined in each language, and memory ownership. It also creates a tight coupling that causes its own problems. There are other challenges such as how to translate Elvish exceptions over the FFI. One relatively recent article that discusses how to do this is https://www.dolthub.com/blog/2023-02-01-embedding-go-in-c/. I can't speak for the project owner but I doubt this will ever be done and certainly not in the near future. There are just too many other enhancements that could be made which would add considerably more value. Rather than attempting to embed Elvish in a different program using FFI it is likely to be simpler to use an RPC mechanism to interact with an Elvish shell.

[IngwiePhoenix]

Oh - I was more talking about something like

import "src.elv.sh/pkg/..."
func main() {
  var context = Elvish.create()
  context.run(...)
}

But I guess I have to dig through the source more and start with cnd/elvish to find that. Still, thanks for the blog post, that was quite interesting!

[IngwiePhoenix]

Dug around a while and came to this:

• An Evaler (...pkgs/eval) and a Parser (...pkgs/parser) are needed.
• elvish uses this to set up it's Evaler:

  elvish/pkg/shell/shell.go

  Lines 99 to 132 in cee682c

  	func (p *Program) makeEvaler(stderr io.Writer, interactive bool) *eval.Evaler {
  	ev := eval.NewEvaler()
  	var errRc error
  	ev.RcPath, errRc = rcPath()
  	switch {
  	case !interactive || p.noRC:
  	// Leave ev.ActualRcPath empty
  	case p.rc != "":
  	// Use explicit -rc flag value
  	var err error
  	ev.EffectiveRcPath, err = filepath.Abs(p.rc)
  	if err != nil {
  	fmt.Fprintln(stderr, "Warning:", err)
  	}
  	default:
  	if errRc == nil {
  	// Use default path stored in ev.RcPath
  	ev.EffectiveRcPath = ev.RcPath
  	} else {
  	fmt.Fprintln(stderr, "Warning:", errRc)
  	}
  	}
  	libs, err := libPaths()
  	if err != nil {
  	fmt.Fprintln(stderr, "Warning: resolving lib paths:", err)
  	} else {
  	ev.LibDirs = libs
  	}
  	mods.AddTo(ev)
  	return ev
  	}

• This here illustrates how to best make a Parser:

  elvish/pkg/shell/script.go

  Line 48 in cee682c

  src := parse.Source{Name: name, Code: code, IsFile: true}

So, whilst there is no "made for embedding" API, it is actually quite easy to set this up - and as it is mostly contained in structures, it should? be threadsafe.

However, there is no way to "return" a value from the script to the parent, as far as I can tell. That said, there is some interesting stuff here: https://github.com/elves/elvish/blob/cee682c489925b47d8d46f9afdc31acc19557d10/pkg/eval/vals/struct_map.go

So I imagine it would be possible to just bind a function to the Evaler that would act as a "return" to the host by wrapping the actual code - which, in the tool I want to make, wouldn't be a big problem, since most of the scripts run are rather short snippets, so wrapping them isn't a big deal.

Would be neat to have a plain pkgs/embed module that simplifies this a little; might make it myself in fact, this doesn't seem too difficult - at least, at present. o.o

[xiaq]

@IngwiePhoenix Glad that you figured out most the stuff :)

The reason you can't get a return value is a consequence of Elvish's language semantics: there's simply no concept of return values in Elvish. Instead, Elvish has outputs, and to "return" something you'll need to write it out with put (for value outputs) or echo (for byte output) or similar commands and capture it.

I've added a doc that specifically answers this question: https://github.com/elves/elvish/blob/master/docs/elvish-as-library.md (beware that the godoc pages on pkg.go.dev takes a while to refresh). The examples for Evaler.Eval in particular shows how to get "return" values from Elvish code.

[IngwiePhoenix]

So if I want a real return value, I would have to execute the snippet and then make the code set a global value which I can then pick off the context - so, something remotely similiar to $?. Hey, works for me - simple enough. :)

Thanks for the additions!