BSA – Integrating PScript

I will need an editor for PScript and the most obvious path is to integrate this into BSA as a building block. PScript evolved from an experiment into being a Plain Interpreter and this makes it easier because I can actually generate source code for other languages from PScript. Meaning you can write PScript and either use Plain, C++ or C# as your target platform – which can be done by either integrating PScript or generating target code.

Plain will change based on the experience with PScript. The VM I drafted for Plain is complicated and I realized that I probably can simplify things a lot by building a Compiler based on the Interpreter. I need to test this new idea a bit later, but the only difference between an Interpreter and Compiler is that an Interpreter execute source code, while a Compiler generate executable code – in Plain’s case I can actually generate C++ or C# etc.

Obviously I can also use PScript as is. This was intended as an Interpreter running embedded and I can easily create a C# or QML PScript component adding HMI libs. For C# I need to create a component that convert between natice C++ code and C#. With Qt I basically need to do the same. While Qt is C++ it has so many tweaks and separate datatype definitions that you end up converting C++ to Qt++. QML/JavaScript Integration comes on top of that.

As for PScript Editor we also need an integrated debugger, but lets take one step at the time. My first PScript editor will not impress anyone, but it comes a day tomorrow 🙂

BSA – Line Moves

To move a line you select it. Selection is done by moving the mouse over the line and clicking the left button. As you select the line the end point markers will show and you move the mouse to an and point and simply move it with the mouse. If you move the marker outside the symbol and release the left mouse-button you delete the line. If you move the marker to a different symbol you establisk a new link with that symbol.

Lines in BSA are automated so you can chose line type, but you cannot change the line except for selecting position and connection of the end-points. This is done deliberately to avoid that you spend time on moving lines around. This is after all a programming tool, not a drawing tool. All links will behave like this, but some symbols will have more restricted connection options.  PLD Diagrams have a number of entry and exit paths that will show up as red or green connection points points. Red means they MUST be connected, Green means they can be left ‘as is’. In both cases you will need to connect the line to that exact connection point. I have done an earlier demo of this that I will use because if you use a diagram within a diagram you need to know what entry and exit points your connecting where. Just drawing a new link to a symbol would in theory create a new entry point – I did not get a good feeling about that one, but lets see.

The picture above show how a line is highlighted as you move the cursor over it. This is a GPU trick. The GPU will allow you to select the actual line, but hit test is very accurate and since the line is thin it becomes hard to select. So what I do is to create a helper line that is thicker to make selection easier. As mentioned only end-points can be moved on a link. I will at some point create an option to ,amually move corners as well, but I preffer automated lines to avoid that the developer use time fidling with lines. The rationale with this is experience with an earlier CASE tool where this was a big issue that used up a lot of time. You control lines by moving end-points and symbols, not by fiddling with line paths.

 

PScript Interpreter – Part 8 – RTOS

We will always have some kind of RTOS behind PScript regardless where we execute, so what I need a few system functions to interface to the RTOS in an uniform way. I my own linear scheduler that scale better than and that is perfect for extending thread shifters. The advantage is that I get the same RTOS functionality on STM32, Windows and Linux etc.

Timers I need to start and stop timer functions. A time function can be something I need to execute on regular basis or simply a function to be executed if I time out on an operation.
Modules I need to be able to start, stop, suspend and continue other modules. The cool thing is that this can be done locally and remotely. And a module on Windows is basically an executable.
Mutexes I need a mutex capability between threads. Events etc will execute as an interrupt and I need a smart way of locking that down.
File I/O Many devices have Serial SPI, TF Card or some way of storing data, so I need an uniform way of accessing this.
AL MCU’s with Hardware I/O needs to be accessed on an uniform way. This is easy as I will borrow a bit of the Arduino design.

I was considering threads, but for that I need a stack so I can as well stick with a capability to execute multiple modules.

I want to re-visit my notes on parallelism, mutex and transactions on Plain. PScript is growing into becoming an Interpreted version of Plain – I have to watch out for bloating/footprint size – adding RTOS, easyIPC etc will cost, but the result will be worth the extra Kb + I can always #ifdef on functionality to keep things down if I need to.

And – finally I think I am at the end. The list above together with modules and events is a bit of a scope creep, but it also makes PScript very attractive to use. I need to return on syntax details on this list.

PScript Interpreter – Part 7 – easyIPC

Part 6 introduces an easy way of communicating where one module raise an event and another receive it. But, for this to work I need some infrastructure that easyIPC provides for me. easyIPC uses a device address where device 0 is NMT (Network Master) on the local network. At this point I don’t care if this is CAN, RS485, SPI, Ethernet or Wifi – the device will have a primary, default network supporting easyIPC or a similar scheme.

This simple drawing illustrate what needs to happen. (1) as myDevice start it register TempMessage and myDevice as a resource to NMT. As myPC start it request that resource from NMT. NMT can be located anywhere in the system, but it is actually on the PC in this case. One challenge is that myPC might request myDevice.TempMessage before it is declared and resources might fall in/out of service. It’s NMT’s responsibility to keep track of this and send a new request answer once a resource becomes available. I want NMT to remember and notify rather based on requests to minimize messaging during start-up. This requires that NMT have a bit of memory – which should be ok.

The second challenge is that I might want two or more myDevice – lets say one for in-house temperature and one for outside temperature – how do I differ the two? The solution is that myPC can extend the resource name with an unique name. And that we in config say what device is what. myDevice and/or NMT need to store this. Once configured we get NMT to re-initialize the network with correct names.

 

PScript Interpreter – Part 6 – Events

I need a way to communicate with the rest of the world, and the method I like is “Events” where you send and receive packets or byte streams. Let’s annotate an example where I read temperature from an ADC once a second and display it on the PC. In this case I use PScript in both ends.

Device code:

module myDevice
    event timer(1000)
        float temp = readAnalog(14)
        raise TempMessage(temp)
    end
end

PC code:

module myPC
    event myDevice.TempMessage(float temp)
        print(“Temperature is “,temp)
    end
end

I introduce the keywords “module”, “raise” and “event” from Plain here. Module is a named application running in our network, while event is an external event.

This is a draft, so I need to work on the syntax details to make this work, but the key concept is that communication should be this easy. (1) I raise and event with parameters and (2) I receive and process the same event on a different device, in this case my PC.

I have a few challenges with the syntax above. (1) how do I handle multiple instances of “myDevice”? (2) how do I differ event filter parameters from call parameters, and (3) how do I send a respond back if I need to.

PScript Interpreter – Part 5 – Footprint

I am currently at ca 13Kb Flash usage and 1Kb SRAM usage. SRAM usage can be adjusted, but Flash usage is fixed and will increase with definition of more build-in stuff. This is on a CISC, so I am not 100% sure how that convert to a RISC MCU, but it looks decently good. I notice that expressions bloat a bit so I might need to do a trade-off and simplify expressions or investigate smarter ways of coding. A typical trick to save Flash is to move logic to tables. I do as an example have a lot of strncmp() in if functions that can be optimized for a lower footprint. But, it is not exactly critical yet. My objectice is below 20Kb. I am aware that PBASIC was implemented in 8Kb back in 1978 using assembly, but this is not BASIC. PScript is actually a full, proper programming language despite the “script” part of the name.

One option that is possible is to extend the Interpreter later to support full Plain syntax as an Interpreter. And to be honest – it is not that far from having an Interpreter to building a compiler. The difference is that in an Interpreter I execute as I parse, in a compiler I need to output the stack as byte code. Lets see. I did not intend to let this grow into a full Plain, but I see the option. I will need far more than 20Kb for  full Plain syntax however.

I am also coding in C++, and my expreience witg GCC on STM32 is that I quickly reach 64Kb. I have suffered Flash starvation on 32Kb and 16Kb devices and been forced to switch to C. I have never investigated the reasons as I believe this can be sorted with some GCC tricks – I have a friend that is more specialized on this that I can ask. But, I think a minimum requirement of 128Kb Flash is ok these days.

One option that I consider is for Plain to skip the C/C++ stage and compile straight to assembly. It is years since I even touched assembly and it’s a huge difference between generating C/C++ code versus generating assembly. The later will require that I generate for every single MCU and handle the differences. But, it is possible. The advantage will be smaller footprint as every automation step will bloat, but the advantages of using C/C++ as a middle step is so many that I doubht it will be worth it. Lets finish PScript and see where we go because this is just an experimental break from BSA at the moment.

A bit back to PScript footprint – I think 20Kb is ok if I can maintain that and it actually looks good – I think I have ca 75% of the code in place and the full Interpreter with comments will be < 2000 lines of code. It will be a nice component to add in on a CLI.

PScript Interpreter – Part 4 – Operators

As with any programming language we need to define a list of operators and how they behave. After years of C/C++ programming I simply copy the list from that language as a start. This first list show mathematical operators.

Operator Description
+ Addition: a+b
Subtraction: a-b
/ Divide. a/b
* Multiply: a*b
+a Positive signed variable/number
-a Negative signed variable/number
% Modulo: a%b
Decrement with 1: –a or a–
++ Increment with 1: ++a or a++
~ Binary invert: ~a
| Binary OR
& Binary AND
^ Binary XOR
>> Bit rotate right
<< Bit rotate left

This next list show boolean operators that only is valid for boolean expressions.

Operator Description
== Equal
!= Not equal
> Greater than
< Less Than
>= Greater than or equal
<= Less than or equal
! NOT
|| Logical OR
&& Logical AND

The trick with this is that I might have 3 operators related to a variable – one pre operator like a sign or ++, — etc, a post operator like ++ or — and a main operator. This needs to be handled by my expression parser.

PScript Interpreter – part 3 – Build In Content

PScript would only be able to perform math unless we integrate it to the target platform. In the drawing above I illustrate (1) the build in function “print” and (2) a variable “a”.

Both are declared as a C/C++ struct in a table that point to a function with a specific format that accept a PSCript stack as input. Thes structs are declared on a separate Flash table in C/C++ so that we don’t use SRAM.

Print is a function so we simply parse the call by executing expressions and putting them on the stack. As print accept an endless number of parameters. Each stack entry have a datatype so in the c function we iterate and convert each parameter to text while printing them out.

The variable a is similar – I call a setget function to either set or get the variable. How this is done is up to C code as it might not be an actual variable at all.

The usage of Flash here is important because I usually have limited SRAM, but plenty of Flash. And as function calls create a stack that is released afterwards I only need to account for max stack deptht in SRAM.

To declare names I use static strings which also are on Flash.

To declare parameters I also use a string. ‘.’ means endless list of parameters. ‘*’ means a parameter of any type while ‘A’ to ‘Z’ are specific data types. This makes it easy to define the functions.

PScript Interpreter – Part 2 – Expressions

I must admit that PScript is a nice break from the larger BSA task. Sadly BSA is written in C# and PScript in C++ because PScript is perfect for extending BSA. But, I can actually use C++ from C#, so lets see.

I have created a stack that I use while I interpret and I am very happy with how that worked out, but I have so far only interpreted the core language and I need expressions. I have done several expression parsers both interpreters and parsers in the past, so what I want goes hand in hand with the existing stack. Let’s annotate an example:

uint32 d
d = 3 + 4 * 5
print(d)

This is a classic mathematical expression test that should print out 23. Let’s interpret this step by step:

  1. “uint32 d” will set up an entry on the stack for the variable d.
  2. Next line “d” tells me that this is a variable and as such an assign operation.
  3. “=” confirms that this is an assign operation.
  4. “3+” get added on the stack.
  5. “4*” get added on the stack. As we now have two or more expression components we evaluate if we should calculate and since * have higher priority than + we just continue.
  6. “5 eol” is added to the stack.
  7. As eol (end of line) is the lowest priority we now calculate “4 * 5” and replace the two last stack entries with “20 eol”
  8. We now evaluate the two remaining expression entries and calculate “3 + 20” replacing the two last entries with “23 eol”
  9. With only one entry left, nothing more to parse and “eol” tagged we are finished. The result is 23. So we assign 23 to the variable d.
  10. “print (d)” is parsed. A call to print and d is pushed on the stack and 23 get printed out.

I have only used a very simple example as we also need to support parentheses and variables in addition to constants. We also need to support functions that return values, but it is all pretty much straight forward.

Once expressions and return functions are supported we pretty much have our own math script where we can combine math expressions and logic for more complex calculations. My experience with expressions tells me that 10 stack entries are a very complex expression. This is no limit, but it is an estimate that we need ca 10 entries (120 bytes) in spare while interpreting an expression. After an expression is interpreted we release the stack.

 

PScript Interpreter – Part 1 – Stack

I surpriced myself with how easy it was to implement the PScript Interpreter and would like to share the basic technique that I am using. The language is still under development, but the idea is to have a minimalistic Interpreter that can be used embedded so I need a very small footprint on the core language. Current tests indicate < 20Kb Flash and 1020 bytes SRAM as a minimum. In addition to that we obviously always have the text – the actual source code that is stored. But, as I target small scripts we actually store a lot in 1-2Kb text. And some of the MCU’s have quite a lot Flash/SRAM available. Lets annotate an example:

func test(uint32 v)
    print(v)
end
uint32 b
test(14)
for b=1 to 10
   test(b)
end

The example above is one of my small test scripts, and to illustrate the inside of the Interpreter I want to annotate stack usage and design. Each statement is a struct with variables and pointers that I need. So as I parse the statements I put these entries on a stack that basically is a 1Kb buffer as follows:

As I interpret “func” I add a func entry on the stack. Func body is parsed without execution.

Next I interpret “uint32 b” and insert the variable b on the stack.

Next I interpret “test(14)” and a call to function test is put on the stack. As this is outside a func we execute the function “test” with 14 as parameter.

What happens now is stack magic. I parse func test again and this time I add v as a parameter on the stack before I assign v the value of 14. v is now a variable with local scope inside the function.

Next I execute test by parsing and executing its body. The next statement is print and I do the same trick of calling print with the value 14. At this point we should get “14” printed out. Stack entry for print and parameter is removed after execution.

The next statement is “end” and as the previous stack entry now is the call to test and it’s parameters I now remove these from the stack and continue on the next line after the call to test. I am now only left with func and var b on the stack again.

Next I interpret the “for” statement and put that on the stack. For is now executed so b is assigned the value 1.

Next I interpret “test(b)” and set up a call to test using the current value of b as parameter. We now execute function “test” as illustrated before and will get the value 1 printed out.

Finally I reach “end” and with “for” being my previous entry I increment and test b with +1 until I reach the value 10. With b=10 I delete the for entry on the stack and continue. As we now are at the end of the script we are finished.

It might be details I need to change, but the key concept of the stack usage means that I can execute quite large scripts on very little SRAM usage. I bviously need sufficient stack to support functions and variables, but this simple technique is already working very well. And with a my MCU’s mostly being 168Mhz monsters it is quite fast as well. That said an Interpreter will always be slow compared to native code.