Basic Syntax

Comments

• Comments can be single line or multi-line
  • Single line comment - // hello
  • Multi line comment - /* hello */
• Comments can nest

/* /* nested */ */

Identifiers

They can begin with _ or a letter and can contain letters, numbers and underscores.

Statements

Statements may be separated by semicolons but it is not compulsory

Blocks

They are declared using curly brackets

{
    let a = 'hello'
    print(a)
}

Int literals

• Int literals can be hex - 0x,octal - 0o
• They can have underscores between them
• Example: 0xdead_beef

Float literals

• They can contain underscores
• Exponential notation is also allowed
• Example: 1.2e9

String Literals

• They can start and end with ' or "
• They can be multi-line
• \t,\n,\r,\0 escape sequences can be used
• Unicode escape sequences begin with \u

  let sparklingHeart = "\u{1F496}" // 💖, Unicode scalar U+1F496
  

• They also allow interpolation using \ followed by an expression in parentheses

  let greeting = 'Hello \(name)!'
  

Types

All values have a type and the type is associated with a class. The primitive types are listed below. These classes have many useful methods that can be seen in the documentation section of the book.

Object

All classes inherit Object. Objects can be directly created using object literals

let point = {x:1, y:2}
// or
let x = 1
let y = 2
let point2 = {x,y}

The point has two properties x and y.

Int

They are 32 bit signed integers. They can be manipulated using the operators +, -, *, -, %, <, <=, >, >=.Whenever any arithmetic operation overflows OverflowError is thrown. Shorthand assignment operators (+=,-=,etc.) can be used too.

Float

They are IEEE-754 double precision floating point numbers. Unlike Int errors in arithmetic operations result in NaN.

Bool

There are two possible values: true and false. They can be manipulated using !(not),and and or. The latter two are short-circuiting(they dont compute the second value if the first is falsey/truthy). They can be used like the ternary operator in C. All values except false and null are considered truthy.

let result = condition and 'true' or 'false'

String

They are a sequence of characters encoded in UTF-8. They cannot be directly indexed but can be indexed using a range.

"Hello, 世界"[7..10] //"世"

The characters of a string can be got using the chars method which returns an iterator. Strings can be concated using the tilde ~ operator. Other types can be converted to strings using the toString method.

Array

They are a list of values. They can be indexed using the [] operator. They can grow or shrink using the push and pop methods. The elements can be iterated using the iter method.

let a = [1,'hello',2]

Arrays can also be sliced (indexed by range) like strings.

[1, 2, 3, 4][1..3] //[2, 3]

Map

They are hashmaps that indexed using any type. The keys of a map can be iterated using the keys method but the order of keys is not defined.

let m = Map{@a:1,2:false}
m[2] //false
m["abc"]=1.5

Symbols

They are like strings but two symbols with the same contents are internally the same object. Comparing symbols are much faster than strings. They are used to store the names of properties and methods for quick access

let cardColor = @red

Range

It denotes a range of integers. The range start..end contains all values with start <= x < end. It is an iterator.

let r = 0..5
r.collect() //[0, 1, 2, 3, 4,]

Null

It is used to denote nothing

let linkedList = {next:null}

Variables and Equality

Scope

There are two types of scope: block scope and module scope. Variables declared in a block can be accessed within that block. Variables cannot be redeclared in a block but can be shadow a declaration from a higher scope. Variables declared at the top level have module scope.

Variable Declaration

Variables can be declared using let or const. Variables declared as const cannot be reassigned.

const VERSION_STRING = '1.0.0' 

Variables can also be declared using 'destructuring declaration'

let point = {x:1, y:2}
let {x,y} = point // x is 1 and y is 2

Equality and Strict Equality

Variables can be tested for equality using the == operator. The == can be used for most situations and strict equality(===) is different in the following ways.

• 1(Int) and 1.0(Float) are not strictly equal
• -0.0 and 0.0 are not strictly equal
• NaN and NaN are strictly equal Strict equality is used by Map.

Functions and Modules

Functions

Functions are can be declared as

• named functions:

  fun add(a,b){
      return a+b
  }
  

• anonymous functions:

  let add = |a,b|a+b
  //or
  add = |a,b|{return a+b}
  

Functions can capture variables

fun makeCounter(){
    let count = 0
    return ||{count+=1;return count}
}
let counter = makeCounter()
counter() //1
counter() //2

Modules

Programs can be broken down into small pieces called modules. A new module is created by creating a new file and can be imported by using the import function. An embedder can decide how to resolve paths while importing.

To share variables across modules they must be exported and then imported by the other module. A module variable that is exported can be accessed using the . operator.

import('math').PI //3.1415926535898

If a module is imported multiple times only one copy of the module will be created and it will be reused.

Example:

//a.np
export fun f(){}
export class C{}
export let x=0

//b.np
const {x,C,f} = import("./a.np")

There are many inbuilt modules that can be used to generate random numbers, evaluate expressions at runtime,etc. They can be explored using the documentation. Functions/Classes that can be accessed from all modules are contained in the prelude module.

Control flow

Basic control flow

Like most languages Neptune lang supports if,if else and while statements.

if x == 5{
    print('five')
}

if cond {
    print('if')
} else {
    print('else')
}

while x!==0 {
    x-=1
}

For loop

The for statement is used to loop through the values in an iterator. Refer the Iterator subchapter in the documentation for more information.

for i in 0..10{
    print(i)
}

Break and Continue

They are used to exit a loop early. Break exits the loop while continue starts a new iteration of a loop.

Switch

The switch statement can be used instead of an if else ladder. Unlike an if else ladder the time taken to go to the required statement is independent of the number of cases. Constant literals must be used as cases of the switch statement. default is used to execute a statement if nothing is matched. The or keyword can be used in a case as shown below.

switch 1+1{
    1 or 2: print('1 or 2')
    default: print('other')
}

Exception handling

Exceptions are used to indicate errors. They are raised using throw. They can be caught using a try catch block.

try{
    throw new Error('abc')
}catch e{
    print(e.message) //abc
}

An error object contains two important fields:

• message - A description of the error
• stack - It contains the stack trace

Error classes can be created by extending the class Error. For example, this is how the class TypeError is defined in the standard library.

export class TypeError extends Error {
    construct(message) {
        super.construct(message)
    }
}

Classes and Objects

Classes are created using the class keyword

class C extends Base{
    construct(val){
        super.construct()
        this.x=val
    }
    method1(){
        return this.x
    }
}

Classes can inherit the methods of other classes using the extends keyword.If no parent class is given then it extends Object. The constructor is created by creating a method called construct. Objects of class C can be created by

let c = new C(7)

Methods of the parent class can be called by super. The this keyword represents the instance of the class. Properties can be get and set using the . operator or be using the [] operator.

let person = {name:'abc', age:29}
person.age+=1
person[@gender]=@male // same as person.gender=@male

Private properties begin with _. They cannot be accessed outside a method.

Tasks and Channels

Tasks are lightweight units of concurrent execution. Tasks are created by the spawn() or spawn_link() functions . A task is killed when an uncaught exception is thrown but can also be killed by the kill() method. If a task fails it does not crash other tasks. spawn_link() links the newly created task with the task that created it. If two tasks are linked then one if one of them dies then the other will be killed. Tasks can be manually linked too. Tasks can also be given names for debugging purposes. Channels can be used to send() and recv() messages. If a task is waiting on a channel, it is woken up once the message is received. The join function can be used to wait for multiple tasks to complete. It throws an exception and if one of them fails.

const {sleep} = import('time')
let task1 = spawn(||{
	sleep(200)
	print('Bye')
})

let task2 = spawn(||{
	sleep(100)
	print('Hello')
})

join([task1,task2])

Errors from multiple tasks can be handled gracefully using the monitor() method of Task. An example is given below.

class MySupervisor{
	construct(){
		this.monitorChan=new Channel()
		this.children=new Map()
	}

    // This method can be called even if it is running	
	// restartPolicy can be
    // @permanent: it should be restarted if it exits
    // @transient: it should be restarted if it exits unsuccessfully
    // @temporary: it shouldnt be restarted
	spawn(f,restartPolicy){
		let task=spawn(f)
		this.children[task]={f,restartPolicy}
		task.monitor(monitorChan)
	}
	
	run(){
		while(true){
			let task = monitorChan.recv();
			let childEntry = this.children[task]
			switch childEntry.restartPolicy{
				@permanent:this.spawn(childEntry.f,@permanent)
				@transient:if task.status === @killed{
					this.spawn(childEntry.f,@transient)
				}
				//do nothing if @temporary
			}
		}
	}
}

All methods of Task can be viewed in the documentation section of the book.

Embedding API

Neptune lang can be embedded in any rust application. The complete API can be browsed through docs.rs. Some key terms that must known are

Embedder Functions

Embedder Functions aka Efuncs are functions that are created by the embedder. The EFunc can push or pop values from the stack using the EFuncContext that is passed to it. EFuncs may be synchronous or asynchronous.

The ToNeptuneValue trait indicates a type that can be converted to a Neptune value. The trait is implemented for i32, String and other common data types.

It can be implemented for any type

struct Point {
     x: i32,
     y: i32
 }

 impl ToNeptuneValue for Point {
     fn to_neptune_value(self, cx: &mut EFuncContext) {
         cx.object();    // push an empty object to the stack
         cx.int(self.x); // push self.x to the stack
         cx.set_object_property("x").unwrap(); // pop self.x and set it as property x
         self.y.to_neptune_value(cx); // an alternate way to push to the stack
         cx.set_object_property("y").unwrap();
     }
 }

EFuncs must return Result where both variants satisfy the ToNeptuneValue trait. The Err variant can be returned to throw an exception.

Methods of EFuncContext like as_int return Err(EFuncError) on error. To return NeptuneError (the Error class of Neptune) or EFuncError we can use the EFuncErrorOr enum.

use neptune_lang::*;
let n = VM::new(NoopModuleLoader);
n.create_efunc("inverse", |cx /*: &mut EFuncContext*/ | -> Result<f64,EFuncErrorOr<NeptuneError>> {
    // pop an int from the stack
    let i = cx.as_int()?;
    if i == 0 {
        // It would be better to create our own Error type and implement ToNeptuneValue for it.
        Err(EFuncErrorOr::Other(NeptuneError("Cannot divide by zero".into())))
    }else{
            Ok(1.0 / (i as f64))
         }
    }).unwrap();

EFuncs can be called using the ecall function in the vm module.

const {ecall} = import('vm')
ecall(@inverse,0.5)  //2.0

Asynchronous efuncs return a Future<Result<T1,T2>>.

vm.create_efunc_async("sleep", |cx| {
    let time = cx.as_int();
    async move {
        sleep(Duration::from_millis(time? as u64)).await;
        Result::<(), EFuncError>::Ok(())
    }
})

Resources

Resources are opaque handles to rust values. They can be freed from neptune using the close() method. They can be created and used only from efuncs. The Resource wrapper struct can be used to return resources

use std::fs::File;
use std::io::prelude::*;
use neptune_lang::*;

 n.create_efunc(
     "file_open",
     |cx| -> Result<Resource<File>, EFuncErrorOr<NeptuneError>> {
         Ok(Resource(File::open(cx.as_string()?).or(Err(
             EFuncErrorOr::Other(NeptuneError("Error opening file".into())),
         ))?))
     },
 )
 .unwrap();
 n.create_efunc(
     "file_read_all",
     |cx| -> Result<String, EFuncErrorOr<NeptuneError>> {
         let mut contents = String::new();
         cx.as_resource::<File>()?
             .read_to_string(&mut contents)
             .or(Err(EFuncErrorOr::Other(NeptuneError(
                 "Error reading file".into(),
             ))))?;
         Ok(contents)
     },
 )
 .unwrap();

Neptune Language Internals

VM Design

Like V8, the VM is a register-based VM that has a special accumulator register. The accumulator register is the implicit input/output register for many ops. This reduces the number of arguments needed. The VM also has many dedicated ops to speed up integer operations like AddInt,LoadSmallInt and ForLoop. The bytecode generated for a function can be viewed by the disassemble function in the vm module.

Value representation

On x86_64 and aarch64 the following scheme is used to represent values.

Empty   0x0000 0000 0000 0000 (nullptr)
Null    0x0000 0000 0000 0001
True    0x0000 0000 0000 0002
False   0x0000 0000 0000 0003
Pointer 0x0000 XXXX XXXX XXXX [due to alignment we can use the last 2bits]
Int     0x0001 0000 XXXX XXXX
Float   0x0002 0000 0000 0000
                  to
        0xFFFA 0000 0000 0000

Doubles lie from 0x0000000000000000 to 0xFFF8000000000000. On adding 2<<48
they lie in the range listed above.

ForLoop op

Many for loops are of the form

for i in a..b {
    do something
}

If hasNext and next methods are called it would be very slow. So two specialized ops exist for for loops of this form.

• BeginForLoop: It checks whether both the start and end are integers and whether the start is lesser than the end.It is only called once.
• ForLoop: It just increments the integer loop variable and compares it so it is much faster than other for loops.

Wide and Extrawide arguments

To reduce bytecode size Neptune lang uses the strategy that V8 does. An op can have arguments of any size. 8 bit arguments are used normally but prefix bytecodes are used for 16 bit(wide) and 32 bit(extrawide) arguments. The Wide and Extrawide ops precede instructions with these arguments. These ops read the op next to it and dispatch to the wide and extrawide variants of the ops. The wide and extrawide handlers are assigned entries in the bytecode dispatch table that have a fixed offset from the normal variants. Macros are used to generate the wide and extrawide bytecode handlers. This scheme has the problem that the number of bytes to reserve for jump offsets is not known. To resolve this problem JumpConstant, JumpIfFalseOrNullConstant and similar ops exist. The jump offset is contained in the constants table. If later it is found that enough space exists to store the jump offset directly in the bytecode then they are converted to the non-constant variants like Jump and JumpIfFalseOrNull and the bytecode is patched. If enough space is not available then the constant table must be patched.

|   AddInt  | |     5     |
|    Wide   | |   AddInt  | |         300          |
| Extrawide | |   AddInt  | |                10_000                      |

Documentation

• Object
• Class
• Array
• String
• Map
• Range
• Float
• Int
• Iterators
• Task
• Channel
• Prelude
• Module 'math'
• Module 'random'
• Module 'time'
• Module 'vm'

Object

All classes inherit from Object

• construct()

  Returns an empty object

  Example: new Object //{}

• toString()

  Returns the string corresponding to the object

  Example: 2.toString() //'2'

• getClass()

  Returns the class of the object

  Example: 2.getClass() //<Class Int>

Class

• getSuper()

  Returns the super class or null if it is Object

  Example: Int.getSuper() //<Class Object>

• name()

  Returns the name of the class

  Example: Int.name() //'Int'

Array

• construct(length,value)

  Returns an array of length length filled with value. Throws Error if length is negative and TypeError if length is not an Int

  Example: new Array(3,0) //[0,0,0]

• push(value)

  Appends value to its end

  Example

  let arr = [1,2]
  arr.push(3)
  arr[1,2,3]
  

• pop()

  Removes the last element and returns it. Throws IndexError if it is empty

  Example

  let arr = [1,2]
  arr.pop() //2
  

• len()

  Returns the length of the array

  Example

  let arr = [1,2]
  arr.len() //2
  

• insert(position,value)

  Inserts value at position. Throws IndexError if position is greater than its length and TypeError if position is not an Int

  Example

  let arr = [1,2]
  arr.insert(0,3)
  arr //[3,1,2]
  

• remove(position)

  Removes the value at position. Throws IndexError if position is greater than or equal to its length and TypeError if position is not an Int

  Example

  let arr = [1,2]
  arr.remove(0)
  arr //[2]
  

• clear()

  Removes all elements of the array

  Example

  let arr = [1,2]
  arr.clear()
  arr //[]
  

• iter()

  Returns an iterator to the elements of the array

  Example: [1,2,3].iter().collect() //[1,2,3]

• sort(compare)

  Sorts the array comparing by function compare.

  Example

  let arr = [4,1,3,2]
  arr.sort(|x,y|x<y)
  arr //[1,2,3,4]
  

String

• construct()

  Returns an empty string

  Example new String() //''

• find(str)

  Returns the position of the first occurence of str. Returns -1 if str is not present

  Example: 'abc'.find('bc') //1

• replace(from,to)

  Returns a new string with all occurences of from replaced by to

  Example: 'abc'.replace('bc','xyz') //'axyz'

• chars()

  Returns an iterator to the characters of the string

  Example: 'abc'.chars().collect() //['a','b','c']

Map

• construct()

  Returns and empty map

  Example: new Map() //Map {}

• contains(key)

  Returns whether key is in the map

  Example

  let map = Map{}
  map.contains(2) //false
  

• remove(key)

  Removes the key from the map. Throws KeyError if the key is not present

  Example

  let map = Map{1:2,2:3}
  map.remove(1)
  map //Map{2:3}
  

• clear() Removes all keys from the map

  Example

  let map = Map{1:2,2:3}
  map.clear()
  map //Map{}
  

• len()

  Returns the number of keys in the map

  Example

  let map = Map{1:2,2:3}
  map.len() //2
  

• keys()

  Returns an iterator to the keys of the map

  Example

  let map = Map{1:2,2:3}
  map.keys().collect() //[1,2]
  

Range

• construct(start,end)

  Returns a range whose start is start and end is end. Throws TypeError if start or end is not an Int

  Example: new Range(1,2) //1..2

• start()

  Returns the start of the range

  Example (1..2).start() //1

• end()

  Returns the end of the range

  Example (1..2).end() //2

Float

• construct()

  Returns 0.0

  Example: new Float() //0.0

• toInt()

  Returns its floor as an int. Throws OverflowError if it cannot be represented as an int

  Example (1.0).toInt() //1

• isNaN()

  Returns if it is NaN

  Example (1.0).isNaN() //false

Int

• construct()

  Returns 0

  Example: new Int() //0

• toFloat()

  Returns it as a float

  Example 1.toFloat() //1.0

Iterators

Iterators can be used in for-in loops. All iterators must have two methods

hasNext: This should return a Bool indicating whether there are any more items left

next: This should return the next element in the sequence

Iterators that extend the class Iterator get the following methods

• each(fn)

  Calls function fn for all items in the iterator

  Example:

  let a=[]
  [1,2,3].iter().each(|x|a.push(x))
  a //[1,2,3]
  

• all(fn)

  Returns if for all items in the interator fn(item) is truthy

  Example: [1,2,3].iter().all(|x|x<10) //true

• any(fn)

  Returns if for any item in the iterator fn(item) is truthy

  Example: [1,2,3].iter().any(|x|x==1) //true

• map(fn)

  Returns a new iterator whose items are fn(item) of the items of this iterator

  Example: [1,2,3].map(|x|x+1).collect() //[2,3,4]

• filter(fn)

  Returns a new iterator whose items are those items of this iterator for which fn(item) is truthy

  Example: [1,2,3].filter(|x|x%2==1).collect() //[1,3]

• collect()

  Collects all items of the iterator into an array

  Example: [1,2,3].iter().collect() //[1,2,3]

• count()

  Returns the number of items of the iterator

  Example: [1,2,3].iter().count() //3

• reduce(fn,initial)

  Applys fn, a function of two arguments cummulatively to the items of the iterator starting with initial

  Example: [1,2,3].iter().reduce(|x,y|x+y,0) //6

Task

• kill(exception)

  Kills the task with exception

• name()

  Returns the name of the task

• setName(name)

  Sets the name of the task as name

• monitor(chan)

  When the task is completed or killed it sends the task to the channel chan.

• link(task2)

  Links the task to task2. If task2 is killed,it is killed and when it is killed, task2 is killed.

• status()

  Returns the status of the task (@running, @finished or @killed)

• getUncaughtException()

  Returns the uncaught exception that killed the task or null if there is no uncaught exception

Channel

• send(msg)

  Sends the message msg to the channel

• recv()

  Consumes a message from the channel.

Prelude

All variables from this module are automatically in all modules

• print(value)

  Prints value

• eval(source)

  Evaluates expression source in the context of the current module and returns it. Throws TypeError if source is not a string and CompileError if there is a compile error.

  Example: eval('1+1') //2

• exec(source)

  Executes expression source in the context of the current module. Throws TypeError if source is not a string and CompileError if source could not be compiled

  Example: eval('1+1') //2

• spawn(fn)

  Spawns a new task with the function fn and returns the created task

• spawn_link(fn)

  Spawns a new task with the function fn which is linked with the current task and returns the created task.

• join(tasks)

  Waits for each task in tasks to complete. If any one task is killed, all tasks in tasks are killed and the error with which that task was killed is thrown.

Module 'math'

• acos(number)

  Returns the arc cosine of number. Throws TypeError if number is not an Int or Float

Example: acos(1) //0.0

• asin(number)

  Returns the arc sine of number. Throws TypeError if number is not an Int or Float

  Example: asin(0) //0.0

• atan(number)

  Returns the arc tangent of number. Throws TypeError if number is not an Int or Float

  Example: atan(0) //0.0

• cbrt(number)

  Returns the cube root of number. Throws TypeError if number is not an Int or Float

  Example: cbrt(0) //0.0

• ceil(number)

  Returns the ceiling of number. Throws TypeError if number is not an Int or Float

  Example: ceil(1.5) //2.0

• cos(number)

  Returns the cosine of number. Throws TypeError if number is not an Int or Float

  Example: cos(0) //1.0

• floor(number)

  Returns the floor of number. Throws TypeError if number is not an Int or Float

  Example: floor(1.5) //1.0

• round(number)

  Returns the number rounded. Throws TypeError if number is not an Int or Float

  Example: round(1.5) //2.0

• sin(number)

  Returns the sine of number. Throws TypeError if number is not an Int or Float

  Example: sin(0) //0.0

• sqrt(number)

  Returns the square root of number. Throws TypeError if number is not an Int or Float

  Example: sqrt(1.0) //1.0

• tan(number)

  Returns the tangent of number. Throws TypeError if number is not an Int or Float

  Example: tan(0.0) //0.0

• log(number)

  Returns the natural logarithm of number. Throws TypeError if number is not an Int or Float

  Example: log(1.0) //0.0

• log2(number)

  Returns the base 2 logarithm of number. Throws TypeError if number is not an Int or Float

  Example: log2(1.0) //0.0

• exp(number)

  Returns e raised to the power of number. Throws TypeError if number is not an Int or Float

  Example: exp(0.0) //1.0

• pow(base,exponent)

  Returns base raised to the power of exponent. TypeError if base or exponent is not an Int or Float

  Example: pow(2,2) //4.0

• abs(number)

  Returns the absolute value of number

  Example: abs(-2) //2

• Constants

  NaN

  Infinity

  E

  LN2

  LOG2E

  SQRT1_2

  LN10

  LOG10E

  SQRT2

Module 'random'

• random()

  Returns a random Float in the range 0.0 to 1.0

Example: random() //0.71189203412849

• range(start,end)

  Returns a random Int in the range start to end. Throws TypeError if start or end is not an Int

  Example: range(1,10) //6

• shuffle(array)

  Shuffles array. Throws TypeError if array is not an Array

  Example:

  let arr = [1,2,3,4,5]
  shuffle(arr)
  arr //[4,2,1,5,3]
  

Module 'time'

• now()

  Returns the number of seconds since the Unix epoch as a float

• sleep(ms)

  Suspends the current task for ms milliseconds

Module 'vm'

• disassemble(fn)

  Returns a string containing the bytecode of fn. Throws TypeError if fn is not a Function or if fn is a native function

• gc()

  Runs garbage collection

• generateStackTrace(depth)

  Returns the stack trace at depth depth. Throws TypeError if depth is not an Int

• ecall(op,args)

  Calls EFunc op with argument args. Throws TypeError if op is not a symbol and Error if op is not an EFunc

• currentTask()

  Returns the current task

• suspendCurrentTask()

  Suspends the current task and adds it to the back of the queue.