CHANGE LOG ============================================================================================= 1.2.0 ===== Remaining experimental features: * lib-tui.go - missing features and further tidying/debug to do * interactive/tab completion mode support for variable depth - needs attention. messy ui and completion selection for deeper nestings * sqlite3 db support Almost out of test (to be supported in 1.2.1): * break/continue if * break for/foreach/while/case * enhanced logging features * function arguments: default arguments * try..catch..then..endtry support * try operator Now considered supported: * flock() call * cmdfallback flag * caret unary operator inside array range expressions. * breaking expression lines on dot char at end of line. * optional name qualification for enums * optional name qualification for structs fixes ----- lots of changes since last win64 build testing: - Windows build compiles. - limited testing done so far with it. Arch build: - added specific build for arch linux - ./build arch - this will act the same as the default build except for linking dynamically instead (for sqlite3 and pcre libs). - this is for when compiling inside arch rather than a cross-compile from a linux platform where *.a files are available. BSD build: - partially tested now. build cross-compiles to BSD and compiles natively. - some issues with kernel/proc/etc missing/unsupported values. - had to supplement the bsd version with some fallback external exec support. Syntactic sugar added for local scope map keys (to deref using a dot char): - e.g.: m["blah"]=42 println m["blah"] println m.blah # both return 42 m.blah="new value" library changes --------------- * (experimental): - added support for sqlite3 db format on local paths - this may be removed later as it adds 4MB to the binary size. - see eg/dblite for example script. * added has_privileges(): checks for root/sudo level active user * added maxuint() : for checking sentinel values (uint64) * updated sort() : now accepts an option map (.reverse bool, .numeric bool, .alphanumeric bool) * added ctrl-l to keypress() (value 12) * table() function for formatting slices of maps/structs as, optionally ANSI-coloured, text tables. - Supports options for colours (using Za colour codes), table width, column widths, alignment, headers, border styles (ASCII/Unicode), truncation, and custom column ordering - Examples: table(data, map(.colours map(.header "[#2]"), .column_order ["name", "age"])) cpu_info(). s2m. table(map(.truncate true, .column_widths map(.Model 40,.Usage 60,.LoadAverage 20), .column_order ["Model","Cores","Threads","LoadAverage","Usage"] )) disk_usage(). table(map(.border_style "unicode",.column_order ["path","mounted_path","used","size"])) - still @todo: add a .hide option ([]string) for curtailing the display of named fields - still @todo: maybe the opposite too with a .show option? or a .showOnlyOrdered to only show fields that also have a .column_order setting? - if no column_order provided then: - map fields as columns are unordered by default - if []struct is provided then the default order will be the struct definition's named field order. - for instanced/anon structs it will be the order they were named in. - that is all over-ridden if a .column_order is provided. * resource usage functions: These should work across Linux/BSD/Windows, but behaviours will be platform dependent (and possibly privilege level dependent): sys_resources() : Returns overall system resource usage sys_load() : Returns system load averages (1, 5, 15 minute) mem_info() : Returns detailed memory information : where available, includes pressure indicators, OOM scores, and slab allocation ps_info(pid,options): Returns detailed process information. options .include_cmdline bool, .include_environ bool ps_tree([pid]) : Returns process hierarchy ps_map([pid]) : Returns process relationships ps_list(options) : Returns all processes. options map: .include_cmdline bool, .include_environ bool cpu_info() : Returns CPU information with per-core data cpu_info([core_number|map(.core core_number,.details bool)]) nio() : Returns network I/O throughput statistics nio(interface_name) nio([map(.interface "device_name", .include_errors bool)]) dio() : Returns disk I/O throughput statistics dio(device_name) dio([map(.device "device_name")]) mount_info(options) : options map: .filesystem type_string disk_usage(options) : options map: .exclude_patterns [string_list] resource_usage() : Returns resource usage for specific process resource_usage(pid) iodiff() : Calculates throughput rates between two snapshots iodiff(snapshot1, snapshot2, duration) * user/group management functions These should work across Linux/BSD/Windows, but behaviours will be platform dependent (and possibly privilege level dependent): user_add(username, options_map) group_add(groupname, options_map) user_mod(username, options_map) group_mod(groupname, options_map) user_del(username, options_map) group_del(groupname, options_map) group_membership(username, groupname, action) # actions are add or remove user_info(username) group_info(groupname) user_list() group_list() keys and their default values: - in user/group add/mod functions: uid: -1 (auto-allocate/current value) gid: -1 (auto-allocate/current value) home: "" (system default/current value) shell: "" (system default/current value) groups: "" (no additional groups) create_home: false - in user/group delete functions: remove_home: false remove_files: false * added pp() function for pretty printing some common types pp(map|slice, [indent], [max_depth]) - TODO: some types not yet supported. this should be expanded so that pp() generates reasonably sane json where possible. * renamed net lib to web * added network library, new calls: These should work across Linux/BSD/Windows, but behaviours will be platform dependent (and possibly privilege level dependent): tcp_client(host, port, [timeout]) tcp_server(port, mode) tcp_send(handle, data) tcp_receive(handle) tcp_close(handle) tcp_available(handle) tcp_server_accept(handle) tcp_server_stop(handle) icmp_ping(host, [count], [timeout]) tcp_ping(host, port, [count], [timeout]) icmp_traceroute(host, [max_hops], [timeout]) tcp_traceroute(host, port, [max_hops], [timeout]) traceroute(host, protocol, [port], [max_hops], [timeout]) dns_resolve(host) port_scan(host, [ports], [timeout]) net_interfaces_detailed() network_stats() ssl_cert_validate(host, [port]) ssl_cert_install_help(host, [port]) http_headers(url, [headers]) http_benchmark(url, requests, [concurrent], [keep_alive], [skip_verify]) netstat() - Returns all network connections across all protocols netstat_protocols() - Returns list of available protocols for monitoring netstat_protocol_info() - Returns detailed protocol information and capabilities netstat_protocol(protocol) - Returns connections filtered by specific protocol netstat_listen() - Returns connections in LISTEN state netstat_established() - Returns connections in ESTABLISHED state netstat_process(pid) - Returns connections for specific process ID netstat_interface(interface) - Returns connections on specific interface open_files() - Returns file descriptors and network connections (lsof equivalent) net_devices(options) - options: .all bool gw_interface() gw_address() gw_info() * added wrapper functions for go-yamlv3: yaml_parse, yaml_marshal, yaml_get, yaml_set, yaml_delete * added zip/unzip support functions: zip_create, zip_create_from_dir, zip_extract, zip_list, zip_add, zip_remove * added a couple more date functions: format_date, format_time * added cron-related functions: cron_parse, quartz_to_cron, cron_next, cron_validate * added some smtp-related functions: smtp_send, smtp_send_with_auth, smtp_send_with_attachments, email_parse_headers, email_get_body, email_get_attachments * Added map utility functions: keys(): Returns a list of map keys values(): Returns a list of map values other ----- * (experimental) Added simple macro system. - this is mainly for use in the REPL - it can be used elsewhere but with certain caveats. - syntax - values must be quoted - macro [!] [-+] m_name [`"] value [`"] - definition macro [!] + m_name [`"] value [`"] - defines a new macro. - if an existing m_name, then redefinition will complete successfully. - use #m_name - searched and replaced before tokenisation - we may need to also limit the places that this can be used. - i.e. repl use is intended. normal script lines should be fine. during eval() or exec() calls? maybe not? inside string interpolation? maybe not? inside system() type calls, e.g. | ${..} {..} &{..} =| et al? - these should be fine but there may be some strange buggy cases doing this. - deletion macro [!] - [m_name] - use macro - to remove all macros - use macro - m_name to delete a single macro - if the ! qualifier is present then a redefinition, deletion or a complete macro set reset will emit a console message. - storage - macros will be held in a global map[string]string - example path="/tmp" macro +options `-l` macro +ez `eza #options --icons=always` #ez {path} - the above will expand before tokenisation so that {...} variable subst can still happen. - alternatively, parameters can be provided as shown below. - added parameterisation - e.g.: macro +add(x,y,z) `$x+$y+$z` macro -add(x,y,z) or, macro -add macro +ls(path) `"eza #options --icons=always $path".nocap.out` * Added 'IF cond' clause to CONTINUE and BREAK statements - just for saving some typing of IF or ON..DO.. statements - BREAK [n] [IF cond] - CONTINUE [IF cond] - cannot be used in conjunction with BREAK CLAUSE * Added ctrl-r support to getInput() for reverse history searching - backspace deletes, ctrl-u clears input - enter selects the current option - cursor up/down find previous/next match - other input amends the search term. * Added permanent history file support for REPL: - History automatically saved to ~/.za_history with 0600 permissions - History loaded on REPL startup and saved on exit (including abnormal exits) - Limited to 255 entries maximum with automatic trimming - Prevents duplicate consecutive entries * Enhanced db_query() function with options map and output formats: - Added prepared statement support via .params option - Added query options: .limit, .separator - Added multiple output formats: string, json, csv, tsv, table, map, array, yaml, xml, jsonl * Map Literal Syntax - Added map(.key value, .key2 value2, ...) syntax for maps literals - Returns map[string]any with the specified key-value pairs - Example: opts = map(.uid 2000, .gid 3000, .create_home true) * added USES clause for try statement. - this is used to dictate 'captured' variables inside a try/catch block. - only captured variables are modifiable from within the inner scope of the try/catch. - they are written back to the outer scope at block end - captured variables must already exist or an error will be thrown. - example: try uses a,b,c throws "test-error" * added ctrl-z (suspend to background) handling for non-windows builds, while accepting keystrokes. * added ctrl-l support to eg/mon * updated some examples, notably eg/mon to demonstrate some of the newer functions. 1.1.9 ===== fixes ----- * updated windows build to incorporate oo/method changes below - compiles and executes - no real testing done yet. * added missing bigi+bigf types to sqr operator * tweaked cosmetic parts of getInput() - DEL key now removes a default string - TAB key now accepts the default string and continues editing from its end. - default string displayed dim+italicised until accepted or varied from. - any content in string overrides the default string - if default string present, and input string empty, then final input is the default string. - while typed text left matches the default string, the default remains visible. - Implemented SIGTSTP signal handling for ctrl-z keyboard input * (experimental) reintroduced break handling (i.e. BREAK [FOR|FOREACH|WHILE|CASE]) - this was disabled in an earlier version due to some suspect test results. - break FOR|FOREACH could confuse each other as they shared the ENDFOR keyword. - this has now been resolved with some additional logic and breakIn now uses the opening token type instead of the closing one. - just checking if the issue still presents. - this cannot be combined with a break count integer. * finally got around to adding short-circuit evaluation and fixes for bool evaluation chaining. * added a few of the missing file and console handling routines for windows. * some minor speed increases in places. * some improvements to REPL line editor: - Reworked getInput() and multilineEditor() for full UTF-8 support. - Replaced all internal string handling with rune-based logic for safe cursor movement and editing. - this still isn't perfect, but its much better. - there's a plan to deal with cursor movement more cleanly over variable- width utf-8/unicode, just not done yet. - Paste handling supports stripping ANSI codes and truncating multiline pastes to single-line input. - Added multi line editor extension to REPL. (ctrl-o) - opens on alternate screen - ctrl-d to accept input, escape to abandon, ctrl-r to return top-line only. - Added dynamic height adjustment in multi line editor. - Line numbering in multi line editor with coloured 5-digit width and truncation indicator (…) for long lines. - tab key generates 4 spaces. not changing this. - basic fuzzy search mode (ctrl-f) - some tweaks to filter out undesirable stuff from pastes. * some refinements to asynchronous calls with ASYNC and await() - picked apart a few small issues with it. some new locking mechanisms introduced. * precedence order tweaked. IN now higher binding power than == , and similar minor changes. * started overhaul of assignment functionality. - split into two paths for current functionality and future use. - current active path retains compatibility with existing test cases - multi-dimensional array/map support added - missing containers in LHS path should auto-vivify. - this will not be worked on further until data type handling has been improved: - move struct implementation away from reflection-based Go-compatible structs towards a map based offering. * updated VAR statement to handle multi-dimension declarations: - e.g. - var ary [][]...[] primitive_type / any - var ary [10][20][4] primitive_type / any - var m map # already worked - var m map[] # multi-dimensional maps library ------- * added to_typed("type_string") conversion function to take an untyped value and convert it to the required type (useful for assigning to VAR type variables / sub elements. - e.g. > a=[1,2,3] > a.kind []any > a.to_typed("[]int").kind []int * (breaking) moved glob_key() functionality into key() - removed glob_key() call * added ssort() : this function is for multi-key sorting of []struct types. - e.g. ssort(ary,["b","a"].list_string,[false,true].list_bool) - new_array=ssort(array,field_name_ordered_list,asc_desc_order_list) - the original array is left untouched. - this is probably terribly slow and janky - lots of unsafe/reflection, use at your own risk. - necessary as Za structs are formed as real Go structs, but without the public/private field name case restrictions. - order booleans are false for descending and true for ascending. * added time_zone() and time_zone_offset() to lib-date. - these return a string (tz name) and an int (tz offset in seconds) respectively. * added editor() which calls into the new multi line editor - usage: content_string=editor("default string",width,height,"title string") * (experimental, testing) added lib-tui.go (text user interface elements) - call parameters and behaviour liable to change after feedback - demo in eg/tui - call list: tui : make an action call tui_new : create a blank action tui_new_style : create a default style tui_screen : switch screen buffers tui_box : direct call to border fn tui_progress : display horizontal progress bar tui_progress_reset : reset progress bar tui_text : display text block tui_pager : interactive text pager tui_menu : present options menu tui_clear : empty the area of the element tui_input : get user input tui_template : convert template string+struct to tui_text output ... - others to be added * error handling library calls - trap("error","func_name(args)") # sets a custom error handler - error_extend(bool) # enables the enhanced error output the following calls are only permitted from within a custom error handler: - error_call_chain # Returns full call chain with function names and arguments. - error_call_stack # Returns function names in call chain. - error_default_handler # Calls the default Za error handler. - error_emergency_exit # Emergency exit with specified code. - error_filename # Returns the filename where the error occurred. - error_global_variables # Returns user-defined global variables. - error_local_variables # Returns variables in the error frame. - error_message # Returns the original error message text. - error_source_context # Returns source lines (before, after) around error. - error_source_line_numbers # lines that error_source_context() refer to. - error_source_location # Returns error location as {file, line, function, module}. * added a couple of calls for gathering exception state info, see the section below. other ----- * (experimental) added try..catch..then..endtry syntax - low overhead system, that executes try blocks in their own subfunction spaces - tracing done through metadata capture. - see the za_tests/ directory for examples - new keywords: try: Start a try block for exception handling catch: Handle exceptions with pattern matching then: equivalent to finally is other languages. endtry: End a try block throw: Throw an exception with category and message throws: Declare exceptions that a try block may throw - new operator: - Added ?? operator for safe expression evaluation with fallback values. - expr ?? - expr ?? fallback_value # usually might be an ex.* exception enum value - Returns fallback_value if expression throws an exception or error - Returns expression result if evaluation succeeds - Supports a variety of data types and nested expressions - Useful for safe property access, division by zero protection, and error handling - Integrates with exception system - failed expressions are converted to exceptions when exception_strictness is enabled - new library functions: exception_strictness(policy): Configure exception handling behavior - policy="strict|permissive|warn|disabled" log_exception(category, message) - Log an exception to the current logging target log_exception_with_stack(category, message, stack_data) - Log an exception with custom stack trace data format_stack_trace() - Format and display a stack trace with colours exreg("name",level): - register a custom exception enum - level is for setting the logging level default during the catch processing. - new syntax patterns: try...catch...then...endtry: Basic exception handling blocks - catch and then are optional. try throws "category"...catch...endtry - Declare expected exceptions catch err is "category" - Match exception by exact category catch err contains "regex" - Match exception by string pattern catch err is "cat1", "cat2" - Multiple categories with OR logic throw "category" with "message" - Throw exception with category and message throw ex.enum_value - Throw exception using enum value - new features: exception enum system with exreg() function for logging level default behaviour configuration. nested try blocks with exception bubbling finally (THEN) blocks for cleanup code automated stack traces with line numbers and function names exception logging integration with JSON and plain text formats coloured stack trace formatting exception strictness policies for different handling behaviors * (experimental) Enhanced logging features: - Added detailed queue statistics showing main vs web access log processing counts - Eliminated parallel logging infrastructure - web logging now uses unified architecture - Enhanced logging status command with queue usage, web logging status, and processing statistics - Web access logs now participate in memory reserve system for better resource management - Web logs now use background queue processing instead of direct file I/O (improves performance) - Web logging now shares rotation, JSON formatting, queue management, and error handling with main logging system - Integrated web access logging with main logging infrastructure - Added log rotation support for web access logs (uses same size/count settings as main logs) - Added enhanced error logging for HTTP 3xx/4xx/5xx responses with status code tracking - Unified JSON formatting between main and web access logs - Added memory management integration - web access logs are dropped gracefully under memory pressure - RFC 5424 Log Levels (8-level system) - Environment Variable Filtering (ZA_LOG_LEVEL) to set maximum verbosity level of logging. - Environment Variable Configuration (ZA_LOG_QUEUE_SIZE) to set logging queue buffer size. - Enhanced Statistics Function (logging_stats()) for more metrics gathering functionality. - Atomic File Writes with locking - on linux/bsd anyway. Windows is shit-of-out-flock. - Improved path validation with better cross-platform support (restricts log paths to more sensible possibilities) - see eg/manual_checks/test_logging for examples - Logging system summary: Logging Statement Types Application Logging: log "message",x,y,z - Primary logging statement (goes to log file + console by default) log level: "message",x,y,z - Primary logging statement (goes to log file + console by default) Basic Control: logging on [filepath] - Enable main logging, optionally set log file path logging off - Disable main logging logging status - Display comprehensive logging configuration and statistics logging quiet - Suppress console output from log statements logging loud - Enable console output from log statements (default) Format Control: logging json on - Enable JSON format for all logs logging json off - Use plain text format (default) logging json fields +field value - Add custom field to JSON logs logging json fields -field - Remove specific field from JSON logs logging json fields - - Clear all custom fields logging json fields push - Save current fields to stack logging json fields pop - Restore fields from stack Web Access Logging: logging web enable - Enable web access logging logging web disable - Disable web access logging logging accessfile - Set web access log file location (default: ./za_access.log) Advanced Features: logging subject - Set prefix for all log entries logging error on/off - Enable/disable automatic error logging logging queue size - Set background processing queue size (default: 60) logging rotate size - Set log rotation file size threshold logging rotate count - Set number of rotated files to keep logging reserve - Set emergency memory reserve for logging under pressure Infrastructure Design Unified Architecture: Application Code ──┐ ├──► Background Queue ──► Processor ──┬──► Main Log File Web Server Code ───┘ └──► Web Access Log File Key Components: Background Queue Processing Non-blocking logging operations Configurable queue size (default: 60 requests, increased for modern systems) Automatic queue overflow handling with warnings Dual Destination System Main logs: Application events, errors, user log statements Web access logs: HTTP requests, service lifecycle, web errors Separate files, shared formatting and processing Format Management Plain text or JSON formatting (applies to both log types) Custom field management for JSON logs Automatic timestamp and subject prefix handling Log Rotation Size-based rotation for both main and web access logs Configurable file count retention Automatic cleanup of old rotated files Memory Management Emergency memory reserve system Priority-based queue management (errors > main logs > web access logs) Graceful degradation under memory pressure Error Integration Automatic error logging for Za interpreter errors HTTP status code tracking for web access logs (3xx=WARNING, 4xx/5xx=ERROR) Enhanced error context with source line numbers Performance Characteristics: Non-blocking I/O for all logging operations Memory-aware request dropping for web access logs under pressure Unified statistics tracking and monitoring Cross-platform path validation and security * Enhanced Error Handling - Added comprehensive error context display with error_extend(true) - Shows function arguments of calls - Displays source code context with line markers around error location - Includes variable state inspection showing relevant local variables - Provides complete call chain with absolute file paths - Maintains extremely low performance overhead on success path - Preserves backward compatibility with existing error handling - Includes recursion protection to prevent excessive output - allows for a custom error handler using trap("error","func") - some new error-related library calls included, listed above. - later version will allow for the errors to chain into interactive debugger use - see eg/manual_checks/test_error_handler for examples * added JSON logging option to LOG/LOGGING statements. - See unused/test_json_logging for example. - similar syntax to USE statement. (- and + for json field addition/clear. push/pop for stacking the field list) * added a small profiler, invoked with the -P CLI option. - this returns a breakdown of parse and execution times for each function called. - this includes the execution times of stdlib calls as well as user defined calls. - the output should be hierarchical, following the stored call chain. - this option replaces a previously undocumented -P call that generated Go trace output. - Separated error reporting stack (`errorChain`) from profiler stack, ensuring no mixing with performance data. - Removed global `callChainList` and replace with goroutine-local tracking (`getCallChain` and helpers). - Updated all `recordPhase`, `recordExclusiveExecutionTime`, and profiler-related calls to use goroutine-aware stack. - Validated correct profiler behavior in normal, recursive, and async test cases. - Preserved error call chain reporting (`errorChain`) as before. - some further changes made to track calls using a context instead of the current horror show that was slowing things down. * added a small debugger, invoked with the -d CLI option. - also added some commands: - DEBUG ON | OFF | BREAK - these enable/disable the debugger at run-time (ON/OFF) or (BREAK) force entry to the debugger. - you may also submit a SIGUSR1 signal from outside of the interpreter to force into the debugger REPL. - use SIGBREAK on Windows. (if supported) - see the script in tools/debug for an example. - breakpoint management and variable watching can be done from with the debugger. - you can also inspect functions and structs. - when debugging is enabled at startup then you also enter the debugger at the start of 'main' and it's exit. - obviously, everything will be substantially slower when in debug mode. performance is likely to be 1/3 of normal. - of course, you should also expect the debugger to be chock full of bugs right now. definitely a few in there. * removed experimental fix, resume statements and try operator - not enough flexibility to be of much use as a trade-off for the performance loss. * added some more examples for the library docs * (experimental) added support for default arguments in function signatures. - normal usage without defaults continues as before - calls can now include defaults: define f(a="123",b=42) end f(b=0) # ... now will populate args with (a="123",b=0) - optional args should be specified in their function signature with f(optarg=nil) - early work, may still have issues with more complex types, needs more testing. * added support for test mode assert error continue with: - ASSERT ERROR f(...) - just using the continue flag in the TEST statement was not robust enough for function call error handling. - the above syntax allows for continued execution past the function failure. - various call sites have been updated to bubble these errors up better - see test script eg/func_args for examples. * USE statement USE - # empties the use_chain internal array USE + name # adds name to the use_chain namespace list (if unique) USE - name # removes name from use_chain namespace list USE ^ name # places namespace name at the top of the use_chain namespace list (pushes rest down) # new name: inserts, existing name: moves USE PUSH # push current chain on chain stack USE POP # pop chain from top of chain stack # push and pop would be used to completely isolate namespacing in a module. - The current namespace is always either main:: or the module name/alias. - If you want to use a different namespace then you need to create a new file and import it with MODULE. - The use_chain array will be consulted, when not empty, on function calls, enum references and struct references for matches ahead of the default behaviour, if no explicit name is supplied 1. explicit namespace (name::) 2. use_chain match 3. current namespace (no :: ref), then 4. main:: (global namespace) - example: # global ns / main program MODULE "modpath/time" AS tm tm::string_date() # call function string_date in module time with explicit alias 'tm' string_date() # tries to call (non-existant) function string_date in main:: namespace (current namespace) # which should error as undefined. USE +tm string_date() # check if string_date() exists in tm namespace and call it if found. # if not found (even though this one would be) then try to call it in current namespace (main::) # which should error as undefined. # whenever there are conflicting names then the first match takes precedence. # i.e. # explicit name > use_chain > current namespace > main * added some support for more array arithmetic: - array = array(a)+b | a+array(b) - array = array(a)-b | a-array(b) - div and mul already existed - todo: add per element array(a) [ + | - | * | / ] array(b) - need to move array concatenation away from the + operator first * added support for functions inside structs. - todo: add skip of assignment back to 'self' if it was not used in a lhs context during a method call. - permitted func definition, setting a parent field to the struct. - changed UFCS in eval_ops.* to: - recognise parent when called as (obj).(method) - ensure (obj) matches type of parent struct - permit calling fn if (obj) encapulates the same field names and types - i.e. all of fn parent's fields must be present in (obj) - if (obj) is a variable rather than expression result, then set some kind of self/this variable local to the func on call which is written back to the (obj) variable on return - on call error, leave (obj) variable in pre-call state - on call, if (expr).(method) do not try to assign self back in caller. - either re-use or strip out remaining 'self' code/structs. - needed to add type information so that self.kind calls show the struct name. - when struct var declared, Kind_override field populated in variable - this is read during method calls, and passed through where required. - (struct).kind overridden when Kind_override is not empty. - added mechanism so that funcs can be attached to structs without unique names. - added a type override when defining funcs (in actor.go) - on dispatch attempt needed to consider the .parent field as part of base/ifs calculation to issue the correct Call() details. - copies of a struct retain the override struct type info - note: recipient must be declared as same struct type as the giver. - e.g. struct q; a int; endstruct var a,b q b=a - added an optional clause to foreach: foreach i [ : type ] in array ... to force a type constraint on the iterator receiving each array element. - this is only really useful currently for saying what struct type to expect, so that its methods carry over to the iterator variable. - see eg/farm for example. - added support for infering type of structs during evaluation to allow method calls to the struct type, when only a single struct field declaration list matches the instance being inferred from. - not doing yet: ary[expr].method calls won't work. not likely to change as it involves chasing back through the tokens and checking the type of ary[expr]. - could fix, but not high priority. see eg/farm for workaround. - not doing: adding access modifiers - this is an, intentionally, very light object mechanism. nothing more complex needed. even this degree of oo is superfluous. there is no concept of private/public. - not doing: constructors/destructors - a struct can already be given default values and instantiated using struct_name() - member variables can be accessed from anywhere in the calling scope. - the unset statement will already undeclare simple variables - not doing: inheritance - more complexity than would ever be needed in this language. - maybe will do: fallback functions if method doesn't exist in object when called. 1.1.8 ===== fixes ----- * added bigi + bigf support to append() and append_to(). - also added some more sanity checking/error info. * added bigi + bigf support in lib-list.go to empty, push_front, insert, peek, pop, remove, head and tail * Added error state for some previously undetected early expression terminations. * fixed bug in last() result in interactive mode * fixed interactive mode bug : module/namespace remnants interfering with prompts * forced 0xnn, 0bnn, 0onn int literals to int type instead of int64 * added support for trailing comma in array literals * set limit in ibase() of 36 for base value to avoid FormatInt() internal limit. library changes --------------- * added set_depth(int) call for setting max recursion depth in interactive context help mode - negative int disables file recursion - 0/1 ignores child dirs - higher ints set child depth * (experimental) added permit("cmdfallback",bool) for interactive mode. - this allows for attempted execution of a failed evaluation as a cli command instead. - it is far from perfect and has some wrinkles still. - mainly included for anyone who wants to use this as a shell - god knows why they would though. * updated help text on permit() call. other ----- * (experimental->supported) breaking expression lines on dot char at end of line. - this is not supported in interactive mode when permit_cmd_fallback is true. * (experimental) added interactive/tab completion mode support for variable depth subdirectory paths. *this may well be broken!* * (experimental) added caret unary operator inside array range expressions. - this returns the length of the parent array, to allow, for example: var a [10]int a[0]=999 a[9]=42 a [999,0,0,0,0,0,0,0,0,42] a[^-1] 42 a[^-^] 999 * tested $ as identifier. - it seemed to be supported, even if just by accident. - may reuse it for something else in future, so.. don't use it! * added bool to int conversion in as_int() : false 0, true 1 * added changes to support more dynamic prompts in interactive mode - see docs/example-startup-script 1.1.7 ===== fixes ----- * [ubuntu,debian,popos] updated apt/dpkg behaviour: - now accounts for previously installed and subsequently deinstalled packages, which leave the package cached. * updated ternary if handling to skip false expressions without barfing unicorn chunks. * added validity check for various regex caching points. (i.e. ~ and similar operators) - an invalid regex (e.g. * would previously have compilation attempted, leading to fatal errors.) - these are now non-fatal in the host language. - however, in most cases they will also be non-fatal in Za also. - this was a deliberate choice, and rvalid() should be used to confirm regex validity where it matters. * same change for regex validity checking in numerous standard library calls, such as grep(), dir(), replace() etc etc - did not change the reg_* functions as the PCRE engine already returns sensible errors. - this may change for consistency later. * added library call for checking syntax validity to lib-string.go - new function: bool=rvalid(string) - this is a wrapper around regexWillCompile() in lib-string.go * fixed bug in field name validation for named struct initialisation. * forced empty source strings passed to pad() to be " " instead. this fixes padding widths not expanding as expected. * the simple tail call elimination had been ever so slightly broken since adding namespace support - i.e. it wasn't working. - current module name was not getting checked properly. should be fixed now. library changes --------------- * added additional error output from stdlib funcs * added term() call to return terminal type string. * added quiet mode to install(string[,bool]) for silent installs * added f2n(arg) call for converting bool to nil (from false) or true - this was added for converting bool results for use with fix/resume. - e.g. with is_open() : see eg/file-read for example. * updated file_mode and stat calls to use lstat instead of stat. * added permit("permit",false) for disabling further permit calls. * added list_fill() other ----- * (experimental) added support for breaking expression lines on dot char at end of line. - this seems to be okay running scripts so far but needs more testing. - it is not yet supported in interactive mode. * removed experimental "self" syntax * removed experimental BREAK on keyword support - has issues with nested FOR/FOREACH combinations. - just stripped it out as of limited use anyway. * (breaking) Renamed WHEN..ENDWHEN statements to CASE..ENDCASE * (experimental) added some more statement aliases: EI : ENDIF EW : ENDWHILE EF : ENDFOR ET : ENDTEST ES : ENDSTRUCT EC : ENDCASE - you probably shouldn't use these.. just testing how they look currently. * no longer considered experimental: - added name qualification for functions - added optional name qualification for enums - added optional name qualification for structs - added expect() call for checking a list against a set of type variants - added syntax for struct initialisation with user values. - code to remove gc bottleneck during recursion * added ctrl-k to line editor (del to end of line) * still experimental: - fix/resume - self syntax - flock() call 1.1.6 ===== fixes ----- * re-enabled SIGINT user handler - removed trapInt main var processing - added trap() lib call for setting @trapInt global instead. - will add other signals to trap() later (SIGHUP/SIGWINCH mainly) library changes --------------- * split lib_file.go source to lib_file_unix.go and lib_file_windows.go - removed flock() call from lib_file_windows.go to get win build compiling again. (didn't work anyway) - will see if there is an easy alternative for flock() in windows later. - most likely to need: LockFile or LockFileEx and an exclusive lock over bytes 0 to size-1 - https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-lockfileex - ignoring for now - just wanted the thing compiling again so that i could test namespace changes. * added interpolate(string) function. * added wrap(bool) function: enables/disables line wrap in non-global display panes. not currently concurrency-safe. other ----- * added '@' as alias for SETGLOB statement token * (breaking change) added name qualification for functions - functions defined in modules are now called using the syntax: module_alias::func_name - this is instead of module_alias.func_name - stdlib funcs remain unqualified - if a user-defined function is called unqualified it must exist in the calling module. - the top level namespace is named 'main' and is added by default at that level if a call is unqualified. - this will break existing code, which is, obviously, undesirable to say the least. - however, this should be the final change regarding namespaces. - now, all enums, structs and functions follow the same conventions. - the only remaining possible change, which is unlikely but you never know... would be to add multiple depths of namespacing, but, should that ever happen (again, unlikely), it would follow the same convention using multiple :: separators and therefore shouldn't break anything. - lib call "exec" has been amended to create calls in the global namespace. this may be adjusted to local later. - lib call "eval" adjusted to use local namespace - string interpolation adjusted to use local namespace * (experimental) added FIX+RESUME statements and ? (try) operator - ? binary operator may be added postfix to expressions in, for example, IF/ON statements (and elsewhere). - on a nil result this will cause execution to stop in the current function (or main). - the FIX statement introduces a code block which is executed in the current function (or main) on receipt of such a nil expression result followed by the ? operator. when code subsequent to the FIX statement is executed then these additional local variables are available: _try_line (int) : line encountered on _try_info (string) : error string - the FIX stanza may be used for cleanup/info operations on this type of failure. - see eg/fix for simple example. - a RESUME command, inside a FIX stanza, will return program execution to the statement following the location of the fault. - if the try operator is followed by a string literal, e.g.: nil?"faultX" ... then a corresponding FIX stanza is searched for that matches the supplied fault code: fix faultX (statements) - FIX stanzas are skipped when there is no current try fault raised within the function (or main). - a FIX stanza that does not contain a code label is always executed. execution will follow through into subsequent code, including other FIX stanzas, if there is no resume or exit (or other form of branch away) at the end of the stanza. - this behaviour may change later. - as usual, if this becomes useful, we may expand on the available _try_* information or functionality. - there are no designs to convert this to any type of exception handling propagating back through the call chain. - it is intended for simple error recovery in situ. * added default string clause to PROMPT statement. ("IS str_expr" as final clause) * (experimental) added optional name qualification for enums - module_alias::enum_name - if provided as unqualified enum_name then it will be internally represented as current_namespace::enum_name - the global level namespace is 'main' * (experimental) added optional name qualification for structs - module_alias::struct_name - if provided as unqualified struct_name then it will be internally represented as current_namespace::struct_name - the global level namespace is 'main' In addition to the above, when we are happy, or at least happier, with their state, then we may look into further changes to support user defined temporary namespace changes in a more programmatic fashion. 1.1.5 ===== fixes ----- * fixed bug in cd: internal dir tracking var synchronisation. * now auto-casting command struct to command output string (.out) when used for a foreach loop range. * updated getInput() for auto-completing only if no paren present. * updated getInput() for continuing func help past paren into args. library changes --------------- * removed fileabs and filebase. replaced by $pa and $pb operators. * (experimental) added expect() call for checking a list against a set of type variants - see eg/expect for example. other ----- * added c-like FOR loop iteration - syntax: FOR [assignment] , [condition] , [expression] # action ENDFOR - any of the terms may be empty - this can be used for infinite loops ( FOR ,, ; ENDFOR ) - FOR ,cond, ; ENDFOR is similar to a do {block} while loop, where the condition is checked at the block end - this is in contrast to the WHILE..ENDWHILE statements, where the condition is checked at the start of the iteration. - the iterator may be specified outside of the loop - Please note! This form can easily be half as fast, or worse, than a 'FOR .. TO [ .. STEP ]' loop - this is due to the condition and expression being evaluated inside Za instead of the host language. - it should still be fast enough for any ordinary requirements of this language. * (experimental) Added syntax for struct initialisation with user values. - struct_name() returns a struct literal with default values (as specified in struct statement for the struct type) - struct_name(args) returns a struct literal populated in struct field order with the args from left-to-right. - e.g. struct person; name string; age int; endstruct person() {name: age:0} instance=person("bob",42) {name:bob age:42} instance.age 42 - anon(named args) allows for an anonymous struct literal with named fields - e.g. x=anon(.age 42,.found true,.name "blah") y=anon(.age 35,.found true,.name "blah") z=anon(.age 42,.found true,.name "blah") z==x true x==y false - added named args in struct initialisation. - e.g. struct person; name string; age int; endstruct new_person = person(.age 99,.name "bob") - N.B. if any value is named then all values must be named. - also added named function call arguments: - this only works with user-defined functions - it DOES NOT work when calling using UFCS-like syntax, currently. - e.g. define swap(first,second) return second,first end swap(.first 10,.second 20) [20 10] - as with structs, either all args must be named, or none of them. * (experimental) playing with 'self' (and faked methods) - nothing to see here, move along. - syntax: struct_instance '.' function_name [ '(' args ')' ] - when first word is a struct_instance name (rather than enum, func, module, etc) - *and* second word is a func name rather than a field name: - a pointer to an amendable copy of the struct is passed to the func as 'self' - the struct_instance is *NOT* passed as the first func arg - instead only the parenthesis surrounded args are passed. - you can modify 'self' inside the func call - however you must return self to be allocated by the caller if you need to modify the source struct. - this is not ever intended to be the start of OO! not going down that path. - just tinkering at the moment to see if this form has any use. - this is extremely likely to be removed - don't rely on it! - if you previously relied on the UFCS-like passing of structs, this may be broken until this is reverted. - apologies for this, but the previous behaviour was undocumented (which is of course my fault!) - the workaround for this is to change the references to the first parameter in func defs to 'self' * updated za.vim * added user aliasing for MODULE command: - new syntax: MODULE str_expr_name [ AS alias_name ] - n.b. str_expr_name must eval down to a string literal whereas alias_name is an unquoted single name. - see eg/long_loop for example usage. - this places the responsibility on the script writer to use aliases to avoid namespace collisions. - the alias name must follow the usual rules for identifier syntax (^[a-zA-Z_][a-zA-Z0-9_]*$) - when no alias is provided the existing rules are used for checking namespace membership. - if a function exists within a module with the same name as the module or its alias, then you must always refer to the function fully qualified. * extended SHOWDEF to permit a module name or alias argument, for listing all definitions from an imported module. * added test output file filename specifier to LOGGING command, i.e.: LOGGING TESTFILE filename_string - this will rename the current/default test file and append further logs to the new named file. - this command does nothing outside of test mode (-t flag). - the default file name is still "za_test.out". 1.1.4 ===== fixes ----- * changed doAssign() so that struct fields can be assigned to in multi-return assignments. - e.g. > struct t_vss; server string; volume int; state bool; endstruct > var vss t_vss > vss {server: volume:0 state:false} > vss.server,vss.volume,vss.state=["sname.fqdn",127,true] > vss {server:sname volume:127 state:true} * added Pop!_OS as a recognised release_id ("pop") * altered UNSET statement to take a comma-separated list of var names * amended INPUT statement to fix a var allocation bug * added type validation to bigi and bigf typed assignments library changes --------------- * amended lib-package funcs to recognise Pop!_OS * added a dup() function for copying arrays/maps * added uint and int as valid types for as_bool() call. * (experimental) added flock() call for application-level file locking in Linux. other ----- * updated example startup script * updated vim file 1.1.3 ===== notes ----- * added IS as a binary operator for type checking: lhs IS (int|uint|float|bigi|bigf|string|bool|nil|map|array|number) -> returns bool fixes ----- * (experimental) remove gc bottleneck during recursion - previously, we assigned a fixed (szIdent) array for variables to each new function environment. - it really needed to be dynamically sized instead. - this allows us to not hit gc anywhere near as hard every single time we call a function. - expected changes were: - change the var ident[szIdent]Variable in a few places, to create a ident *[]Variable instead. - add a make() call for the ident somewhere in Call() before it is required (probably just before functionArgs need populating.) - amend vset to bounds check and reallocate if ident[bin] is greater than len(ident). - maybe change vget too, so that it doesn't try to read from above the length. - hunt down the other places where we assign to ident directly (e.g. endfor iteration) and shove through similar realloc process (or push through vset/vget instead) - ... and possibly other things needed changing too (like all the function declarations in each library file for ident[szIdent]Variable, only about 350 of them to do!) Update: all of the changes above have been done. Still testing, but recursion is now considerably faster. It has slowed down normal var allocation a little as we have to bounds check and realloc ident arrays dynamically now, but not a big drain so far. There are two consts, identGrowthSize and identInitialSize which control the process. The exception is global and sysvar storage which are now initialised at len 64 by default as they are likely to require more variable space anyway. * added some dummy funcs and struct changes (for stat()) to allow windows build to at least compile again! library changes --------------- * s3sum(filename[,blocksize]) added for calculating checksum of local file. - this and other *sum funcs have been moved to separate source file. * BREAKING CHANGE!! renamed float, int, uint, bool, bigi, bigf, string functions with prefix "as_" - SORRY! - this was done to accommodate float, int, uint, bool, string, bigi, bigf as type keywords other ----- 1.1.2 ===== notes ----- Features now considered non-experimental: - arbitrary precision numbers - fsnotify support (linux) - SYM_BOR unary precedence - identifier reference lookup (-V) - result chaining - string OR string operator - VAR statement changes - single level module namespaces - short string operators: $lc $uc $st $lt $rt Features almost out of test: - clamping operator - not enough robustness checking done yet - libpcre3 wrapper (linux) - may add some more calls from libpcre as quite limited currently - exec() - still some misbehaviour when exec string contains comments or string interpolation - also not reporting detailed enough locations on error - BREAK on keyword support (avoid this for now) - some more tweaks done recently to BREAK and BREAK , but not thoroughly tested yet. Features still considered experimental: - uninstall() stdlib call (linux) (doesn't cover enough OS) - logical OR auto-casting of ints and strings (needs more feedback) Anything noted as experimental below, not explicitly mentioned above should be considered as remaining experimental. library changes --------------- * added big int and float support in abs() * (experimental) added exec() for evaluating code blocks. can be blocked using permit("eval",false) - added return value processing, e.g: a,b=exec("for e = 1 to 10; endfor; return e,e*2") # a==10, b==20 - will possibly add a separate permit() key for this action. * added some additional type conversions to some of the list_*() functions * ftell() and stat() file functions added. - ftell(): returns file pointer position in open file handle. - stat(): returns a struct of more detailed file attributes. - e.g. >> stat("/tmp") &{Dev:2049 Ino:1048578 Nlink:18 Mode:17407 Uid:0 Gid:0 X__pad0:0 Rdev:0 Size:20480 Blksize:4096 Blocks:48 Atim:{Sec:1640215603 Nsec:898341080} Mtim:{Sec:1642869189 Nsec:805359276} Ctim:{Sec:1642869189 Nsec:805359276} X__unused:[0 0 0]} >> stat("/tmp").Nlink 18 - these were in the previous version, but only now documented. * added fflush() file function. - write buffer flush. - not previously present. equivalent of (*os.File).Sync() in Go. * added md5sum, sha1sum, sha224sum and sha256sum library calls. fixes ----- * occasional name binding issue in interactive mode fixed, hopefully! * testing a change to interpolation nesting in eval.go * damped ansi output from bg/fg 256/rgb funcs. * fixed one of the issues in -r mode. * had an issue with some scripts requiring coproc(false) when piped out. - Doh! fixed. - was setting an alias for multi-column ls width at startup/sig_winch resize to the term_width. - termwidth (TW) defaults to -1 when running without an attached terminal. - the ls was failing with an invalid -w option from the alias. - changed to not create the alias when TW==-1. - at least it's not as catastrophic as i thought! - also added support in -> (map) operator so dir() could be used instead of ${ls -1} in eg/ps. * post test rebuild with freebsd 13.x + go1.18.x: - updated interactive mode ls alias for freebsd/bash - updated build script to remove error in upx detection. - otherwise, freebsd build appeared to be fine, against very limited testing. other ----- * (experimental) added startup script processing. - a startup script can now be located at $HOME/.zarc - this script will be executed only in interactive mode. * (experimental) added dir/filenames back into tab completion in interactive mode * (experimental) added {...} and ${...} token pairs for delimiting system command lines - e.g. { echo "test" } returns a struct containing: - .err : stderr ("") - .out : stdout ("test") - .code: status code (0), and - .okay: bool confirming exit status of last shell command in the {...} construct - ${ echo "test" } returns a string containing just the .out field from above. - &{...} async block execute - returns a struct containing id and handle to the block execution. - these can be discarded for anonymous background jobs - stderr and stdout are hidden for these. you have to take the results, as indicated below, if you want those or the status code. - if you wish to collate the results, then the id and handle must be assigned to a map and parsed using the await(ref mapname,block_flag) call. - the same struct as for {...} is returned in the map value of map[id] after the await() call if the call is complete, (otherwise nil). - the block_flag must be 'true' to wait for the results of all map entries, otherwise each map entry's job handle is only polled once. - see eg/blocks for an example. - these blocks are expressions, and can be used as such. e.g.: on {ls -l | wc -l}.out.int > 2 do print "more than 2 files" * reworked name binding routine. much faster now. had to get rid of l.h.s. name interpolation again though :) * added uint32 and uint64 as available types for VAR. - added aliases of ulong and uxlong respectively. * added float*[]float and int*[]int support. these return []float and []int respectively. - multiplies each element * added float/[]float, []float/float, int/[]int and []int/int support. these return []float and []int respectively. - divides each element by other operand or operand by each element - element divide by int/float zero returns error - int/float divide by element of zero, returns zero - these were added mainly for scaling sets of values. - if there is a requirement for similar operations using addition and subtraction they may be added. * added dir entry support to filter operator ?> - may add other struct types later, depending on performance. it's another hack job. - e.g.: dir() ?> `#.name ~i "^lib-" and #.size<3000` [{name:lib-pcre.go size:2334 mode:416 mtime:1648825572 is_dir:false}] * (experimental) added clamping operator: - this re-uses the subscript/range operators: v[start:end] ... but changes form when v evaluates to a number. start and end are optional when present, they clamp v to the floor and/or ceiling specified in start and end. the returned type is either that of v, start or end depending on the clamp action taken. - e.g: >> for e=-3 to 3; print e[-2:2]; endfor -2 -2 -1 0 1 2 2 >> 3[pi():10.2] 3.141592653589793 >> 11[pi():10.2] 10.2 - illustration of type change from v to end type: >> 9.4[1:9] 9 >> kind(9.4[1:9]) int 1.1.1 ===== library changes --------------- * for arbitrary precision support: - added bigi() and bigf() for converting to big int and float from other types. - added list_bigi() and list_bigf() for converting to []*big int and []*big float from []interface{} - string(s[,prec]) conversion updated for outputting bigi and bigf with optional precision. fixes ----- * Moved cmdlock deeper into call stack for linux/windows shell/exec calls - locks on Copper and GetCommand calls instead of system now. other ----- * (experimental) started arbitrary precision number support: - basic type support for big int (bigi) and big floats (bigf) - support in structs underway (bigi,bigf): - can define bigi and bigf types in struct definition - can assign to these types as normal and read from them with dot operator - fixed default values for bigi/bigf fields - @note: also fixed an outstanding bug when assigning default to multiple vars - base operator support (+,-,*,/,%) - ev_add, ev_sub, ev_mul, ev_div,ev_mod (for bigi) updated - not doing for now: modulo operator for big.float - type errors instead - added type support in == >= <= > < != operators - fixed "in" operator for big int/float - amended eg/pi and eg/fib.tailrec for testing - eg/fib.tailrec looks fine now calculating arbitrary lengths - only change needed was to cast the input ints to big ints with bigi() - added power operator (**) support for big int type. errors with big floats. - added sqrt operator support. - this only returns the new value and doesn't set the passed variable - this was to keep the sqrt op in line with other types' behaviour - not too worried about the extra allocations/reduced performance from this - works with both bigi and bigf - added pre/post dec/inc support for bigi and bigf - fixed missing default 0 for bigi/bigf in struct fields: - slice support done, hopefully! - same for loop constructs. - must declare big int/float with VAR or a literal, - e.g. var p bigf ; p=pi() - ... obviously, that would be rounded to uint64 precision at best, as pi() returns a constant - or, another_pi=3.1415926535n - if declaring as a literal - Please Note! big numbers are slow. there's a lot of memory allocation going on in the background in order to integrate them the way we have. this is unlikely to be improved. they are, as usual, just a convenience feature. most people will not need them most of the time. * added support for 'n' suffix on big int and float literals. type is determined by presence of decimal point. * added underscore stripping in INPUT .. PARAM|OPTARG pos fetch "is number" check * added IS literal separator in INPUT .. PARAM|OPTARG .. [ IS "description" ] - was to fix broken expression handling in "pos" immediately before optional IS * added operators for path strings: - i.e.: $pb for base file name $pa for absolute path $pn for base name without extension $pe for path extension $pp for parent path e.g.: cd("/") a="tmp/test.file" $pb a # -> test.file $pa a # -> /tmp/test.file $pn a # -> test $pe a # -> .file $pp a # -> /tmp 1.1.0 ===== library changes --------------- * experimental: added support for libpcre3 basic use in musl/libc builds - wrapper around some functions in github.com/GRbit/go-pcre - have to install libpcre3-dev on the build machine - need an alternative for windows builds, as they are dynamically-linked. - only added these calls so far: - reg_match(haystack,pcre_needle) (bool) : match present? - reg_filter(haystack,pcre_needle) ([][]int) : list of matching pair positions - reg_replace(haystack,pcre_needle,replacement) (string) : replace all occurences - as a test, eg/regex (benchmarkgames test): - was previously taking about 16 seconds with the 5MB test file - directly replacing the filter and replace calls with reg_filter and reg_replace in eg/pcre_regex takes the completion time down, on the same machine, to approx. 3.254 seconds - once we have time to test this more and find a workaround for other OS, this will become a fixed non-experimental call, although we may wait to see if the go regexp handling receives any further improvement first. - there are alternative go-pcre packages that work with windows, but they would need some work to wrap the calls and the search flags differently and i have no idea how they perform yet either, so not rushing to fix this for windows. * experimental: added basic support for fsnotify golang lib - new lib funcs: ev_watch, ev_watch_close, ev_watch_add, ev_watch_remove, ev_exists, ev_event and ev_mask - see eg/notify for example use - tested okay on ubuntu and windows so far - windows uses normal file paths, e.g. w,e=ev_watch("c:\users\daniel") - you may need to execute Za with escalated privileges depending on the paths you want to probe. * added clean() for stripping nested {} braces from a string (for sanitising input) * added filebase() for returning the base name of a file path. * added inset() for offsetting a string by N chars from l.h.s. * added tab (7) and shift-tab (6) support in keypress() * added shift-up (211) and shift-down (210) support in keypress() * modified logn to return 0 on logn(0,n) instead of +/-Inf - logn has mainly been used for graphing/series processing and such results would manually be converted to 0 anyway. - this isn't great behaviour, but will stay this way until a better alternative presents itself. - if correct behaviour is more important then ensure you don't push a zero in to logn to begin with. * removed references to globlen * checked lib-os for windows differences: - some calls have no direct equivalent, so ignoring those. - however, we did overlook parent(), fileabs() and filebase(), so added those for windows * mem_summary() and dump() artifacts removed. * added bg256(n) fg256(n) bgrgb(r,g,b) and fgrgb(r,g,b) for expressing 256 colour and RGB ANSI colour codes in the background and foreground. e.g. print bgrgb(0,127,0),"MEDIUM GREEN" fixes ----- * added support for [][]int in FOREACH, len() and various other places. * windows: fix scrolling + term height issues. - partially done. still some artifacts, but better than it was. * fix help highlight colour / win - turns out, it was fine, just not good palette selection. changed to a red background for selection in windows. * windows: filepaths slash and backslash quoting clarification - checked: added escaping back in for backslashes \\ - backslashes are fine if captured as part of a string token, - test fix: escaping backslashes in .Original and/or borcmd in phraser.go - test fix: added SYM_BSLASH symbol support in lexer.go - this is to prevent unquoted backslashes from being thrown away. - currently, as in linux build, only backticks are supported in | command form for wrapping cmd. - in expression form double quotes are also supported - should probably unify and document all of this! will see how it goes for a while first - more changes may be required. * checked associativity of | string_expr form. - as an example, something like this: | "which upx | wc -l" . trim(0) . int ... required parentheses around the | "string" part to correctly associate: ( | "which upx | wc -l" ) . trim(0) . int - experimental: special cased the unary SYM_BOR precedence value, in eval.go/command(), to 65 instead of 0 - SYM_DOT prec is 61. - this seems to resolve the parentheses requirement, but checking for other issues before calling it good. other ----- * added support in lex.go for hex prefix numeric literals, e.g. 0xf000 => 61440 dec - hex processing takes precedence over float tokens (i.e. 0x0f is 15, not 0.0) - hex numbers are int64 * also added similar for binary and octal (0[b|B] and 0[o|O] prefixes) - updated eval_ops.go / basic math operators to work with int64 better * default build is now a unified build for use on both alpine and other linuxes - the old separate builds are available using ./build alpine|libc * updated za.vim * added -W startup flag: - this produces a definitive exit when addition is performed with string and either int, float or bool - this may be extended where appropriate if it proves useful. - the flag is intended to help find unintentional type mixings when debugging. - ideally, we would catch this sort of thing in a type system, but as we don't exactly have one, well.. you get this instead for now! - you will probably only use this if you are setting out to enforce types throughout a piece of code. - for most uses of this language, it just shouldn't matter, but it was computationally cheap to add this check, so it's in. * added support for optional printing at end of AT command: - e.g. AT 10,20, 1+2, " = 3" - LF at end-of-line must be user supplied. for AT the default would usually be to not require it anyway. * also, unified PRINT, PRINTLN and LOG (and now AT) into a common call. * decision: ruled out support for prepared statements. - to do this right would also mean supporting transactions and probably at least a couple of db engines. it also may give the impression that it's a good idea to use db queries in Za just because some support is there. the existing sql support was only ever meant as an emergency measure. - any program that requires statement preparation, transactions or other db engines should already have been getting written in a more suitable tool for security, performance and maintenance. - this may change in a future version, but there's already enough other stuff to get right first. * added case to ignore assignment in lvalue to terms whose identifier is the underscore. - e.g. a,_,c=["a","b","c"] 1.0.16 ====== library changes --------------- * Removed lib-ui and supporting code. - was an experimental feature. it worked and is still available in 1.0.15 but it will not be included going forward. it was simply too heavyweight a feature to support for the occasional use it would receive. * added permit() types to block use of shell calls and eval(): - permit("eval",) and permit("shell",) both default to true. - set these (to false!) at your own discretion if you are exposing a web server or similar. - scripts should stop dead when these trigger. we'll probably add something to trap these if we ever improve the error handling. they are just a (very minor) safeguard for now and should not be relied upon for all of your security. * added has_term() and has_colour() library calls. * added sinh, cosh, tanh, asinh, acosh and atanh math library calls. * removed some hard exits in unmap(), key() and globkey() * added eqlen() lib call in lib-list.go: checks that length of lists in a list-of-lists match: - e.g. eqlen( [ [1,2,3] , [2,3] ] ) equals false - e.g. eqlen( [ [1,2,3] , [4,5,6] , ["one","two",3] ] ) equals true - also added support for list of strings. e.g. [ "one", "two", "three" ].eqlen => false fixes ----- * reviewed the code around ASYNC/AWAIT for problems. - it's working fine, just not all that clever. - absolutely, under no circumstances, ever try to write a recursive function that has async inside of it. it will work, but it's awful slow. - recursion is not exactly fast to begin with and you lose most chances to take the small advantage of tail-call elimination when using async. - also passing the gc flag creation for async routine to await instead of task() - this could result in some async calls not getting their id reused, but otherwise helps to preserve the return results for longer. - should fix this properly eventually - could do without passing the results over a channel back to the callers. * Added a gc flag to calltable entries so that dead entries can be periodically reused in the function space allocator. It doesn't cull or reorder anything, just marks for re-use. - this should have quite a large effect on memory use in long running programs w/many fn calls. * Change implemented for determining start position of SYM_BOR ("|") commands better - this was previously making a bad estimate of start position from source code line string - now recording the position of the SYM_BOR token in input source in order to store commands in a separate record when present. - this was mainly a problem in " ON..DO | cmd " where the .. expression may have had a | symbol in it. - also tweaked this for |, =| and =< as an expression * Updated trapInt and general handling around interactive mode main ident id. * fixed: removed grep/cut calls during startup. using internal funcs instead. * fixed: interactive mode / post-inc/-dec triggers false negative in global var name * fixed: setglob post-inc/-dec. * now recognising [] of types and interface{} as a type in struct read/writes from/to file, (in actor.go/s2m and actor.go/m2s). - also now auto registering the basic struct field types in gob.Register. * added []any type as valid list type in esplit() - this was forcing you to cast a literal list like [1,2,3] to something like [1,2,3].list_int before passing it through esplit(). * added bottom margin setting for interactive mode - pre-emptive inclusion so that we can add better interactive help later. * poked around a bit tidying tab completion output. (interactive mode) - moved from rhs of input to lines beneath instead. - fixed some of the random seeming cursor positions after completion selection - still needs more bug whacking. - added func desc when single func remains in completion text. - added input parameters to help text. - made context help move as left/right pressed on "selectedStar" changes. - now recognises a dot (.) as a word separator in context help - removed ... in help options. - a fuller list is now displayed (up to 30 entries) - to accommodate this, the bottom input buffer margin has increased to 8 lines. * removed 'nassoc' type, to reduce complexity. * added out-of-bounds and type checks in eval_ops.go/accessArray for sub-scripts. * fixed bug in enum_all() and enum_names() (crash on invalid enum name filter) * updated za.vim * revised console_*.go files: - moved getInput routine to console_common.go - added some dummy funcs in win/linux where they were missing - added a "never pasted" result in getch() in windows source. language changes ---------------- * (experimental) Added -V option for finding identifier references. * (experimental) Added some auto casting in logical OR checks: - ints convert as 0:false, not-zero:true - strings convert as "":false, not-empty:true * (experimental) Added an explicit string OR string test: - returns l.h.s. string if l.h.s. is non-empty - returns r.h.s. string if l.h.s. is "" - this allows for setting string defaults, for example: region=get_env("AWS_DEFAULT_REGION") or "eu-west-1" - may also add similar for ints if the above doesn't prove bad in some way. * Added break level count to BREAK statement. - arguments treated as expression - must eval to int. - sets the number of FOR/FOREACH/WHILE/WHEN levels to escape to. - e.g. while for e=1 to 20 break 1 endfor endwhile - ... "break 1" escapes to between endfor and endwhile - ... "break 2" escapes to after the endwhile. - ... "break 0" or no argument acts like break 1. - (experimental!) added "break [for|foreach|while|when]" to make break skip to the outside of the next matching construct type. i.e. from example above, "break 2" == "break while" a="two" when a is "one" print is "two" for e = 1 to 20 break when endfor is "three" print endwhen 1.0.15 ====== fixes ----- * dot operator binding more tightly to lhs string than $in op is to rhs string - should be fixed now (extra case with prec. 70 for $in operator in unary switch) * Changed var backing store layout and name binding scheme. * replaced callstack id+space allocator. - using a sequential modulus fn instead of random numbers now. - needs cli opts for setting gnfsModulus constant - needs cli opts for other related allocator tweaks - needs to actually deallocate reservations too! - currently, mem use grows to a max (related to MAXFUNCS). - it should keep working at this depth without further allocation however, the callstack doubles in size on demand up to that max. - this didn't happen with the old allocator, so still needs fixing. - this was needed as the old allocator had a couple of problems: 1. random number use, even using faster rng involved calling sys.time and other hacks. 2. check for next available slot was basically counting up from zero looking for next instead of starting at around the right place. - had to rework a few things around locks, parser instances and lhs/rhs ident tables - less small allocs + gc, but still plenty that can't be avoided because of dynamic typing parts of the code. will just have to live with this. * added lookahead caching (stored in ast token for C_If) on IF statements. - currently only for IF..ELSE..ENDIF lookups. ensures the lookahead only happens once. * fixed trapInt handler (ctrl-c) - needed repointing to main ident space post changes above. - it was also pretty broken since we added multiple return values. - ... and was using the wrong parser in the interrupt handler. - should be okay now! - still want to add other event handlers, mainly for sig winch to begin with. * localised loops[], depth[] and lastConstruct[] - this allowed us to remove *a lot* of mutex locking. these were previously globally shared as we had thoughts regarding function re-entrance. if this still happens, we'll find another way - the shared storage was too slow. - we could possibly move these entirely into the parser struct later if needed to permit some kind of yielding facility. * moved the program counter from local var to each parser's struct. this saved us needing to take copies of it for error reporting, but otherwise a negligable speed change - just less GC. * added timing checks at debug_level>10 for lex/parse-to-ast of global, functions and modules. * moved binaryLed inline manually to dparse function. saves a function call which was happening far too frequently. * removed siglock mutex. * fixed mutex unlocking in C_Assert - could double unlock when recursed through with user-defined funcs. * the removal of the mutices above and other changes have contributed to around a 20% speed up in eval and general statement processing speed. * added identifier subtype and check for builtin const/library calls in lex.go - this should speed up some types of code a little. was previously always checking these during identifier lookups in evaluation. it also picks up user defined functions if they have already been encountered by this lexing step. * stripped out caching in lex.go/getNextToken() as it offers no real gains any more. * updated za.vim * fixed reverse() on []string. language features ----------------- Added some unary string operators. The following operators are currently (and experimentally) enabled: - $lc "string" : to lower - $uc "string" : to upper - $st "string" : string trim (whitespace "\ \t\n\r") - $lt "string" : left trim ( " " ) - $rt "string" : right trim ( " " ) - example: >> q=read_file("/usr/share/dict/words").split("\n") >> foreach a in q ?> `#.has_start("d")` -> "$uc #" >> println a >> endfor * added assign command output operator =< - equivalent to =| except the string output is forced into lhs assignment instead of the full struct. * changed unary file input operator to $in expr_filename_string (was <- expr_filename_string) * added file out operator (expr_string $out expr_filename_string) * added underscore as a formatting separator inside numeric literals (both int and float). - i.e. 1_000_000 == 1000000 (int) - 1_000.1_23 == 1000.123 (float) - 1_000_f == 1000 (float) - the underscore is removed immediately during tokenisation, so will not appear in debug output. * added NaN as a valid float state and constant * added NaN equality check (e.g. sqrt -1 == NaN) (doesn't follow ieee spec, but may be useful?) * unary file input operator ($in) now removes single trailing 0x0a char from input file. - this keeps it in line with the cat shell command, but may change further. library features ---------------- * added some more lib calls: - os: can_read, can_write, parent, fileabs - conversion: dtoo, otod - os: is_symlink, is_device, is_pipe, is_socket, is_sticky, is_setuid, is_setgid - these all require a signed integer (int) as parameters. i.e. int(file.mode) from dir(). * updated info for ibase, wininfo, int64, zainfo, byte, floor, ui_w, time_*() lib functions. * added varbind(name) to fetch the variable binding id for variable "name" - for debugging. * updated kind() to write float64 types as "float" * added grep() as alias for line_filter() * added stripcc() lib call. (remove za colour codes from string) 1.0.14 ====== fixes ----- * updated vim syntax file * added 'f' suffix on numeric literals for enforcing float type. * library function name review (for consistency) - breaking changes: some function name changes have had underscores added/removed for consistency. * added expect_args() to all standard library functions where it makes sense to. * added { and } as outer delimiters for ENUM as well as ( and ) * altered ENUM to accept expressions in member assignments. - this can contain circular references to earlier members such as: > enum colours { r,g=10,b=colours.g*3,w } > print ref colours.enum_all [1 10 30 31] * updated append_to() help * added support for reversed count ranges in .. operator. e.g. 20..5: - eval_ops/ev_range() amended. returned ranges are always int-based. - range start and end may be specified as uint(64),int(64) or float, but types of start and end must match. - floats should be specified in parentheses where necessary to avoid confusion. * finally got around to adding a type check on struct build. - currently permitting: bool, int, uint, float, string and mixed. * altered PROMPT to permit validator regex to be built from an expression instead of a literal. * changed windows default ansi setting to true. * removed -l option. * added SHOWDEF back in. the output is occasionally a little off due to semicolons, but still has some use. * fixed ui_cursor_visible() input type check. * oops, fixed issue when auto-allocating from size 0 in array assignment. (sz test in eval.go/vsetElementi func) * Fixed stuck FOR loops when loop var already exists with different type (actor.go/C_For) - now throws an error report when var already declared. - *except* when var is an int - this highlighted an issue around whether loop variables should be disposed of post-loop: - as it stands, sequential loops need to use different loop var names so that we do not have to dispose of them when a loop ends (which can be useful). - this will most likely only occur when a FOR loop tries to use the same var name as a FOREACH loop in the same function block. - it's probably fine as it is, but should review soon. (now done, all okay) * struct default values: - You may now supply a default value inside a struct definition for a field. - Format: struct sn; a int [ = value ]; endstruct - Example: struct person first_name string surname string age int = 18 endstruct - on initialisation with VAR the defaults are set. - this is currently only implemented with scalar creations of struct in VAR. - still need to add default value support in, e.g., var people []person - should just be a virtual copy/paste job, but testing current implementation first. * added map[string]interface to sort(). sorts on key instead of value. * added pgup/pgdown to wrappedGetCh() for use in keypress(). (dec 15/dec 14) - untested in windows. windows is also missing copy/paste functionality. language features ----------------- Several changes are underway to support windowing features through opengl. This will be done with a subset of the Pixel Go library. PLEASE NOTE: the default version currently only works (correctly) on Linux with certain libraries installed and a particular level of OpenGL support (v3.3). (3.1+ if you hack it a bit) : required support and workarounds for opengl(es)1/2/3 can be found through the wiki pages : for pixelgl, glfw, etc. : we managed to get this working on a deliberately feeble test rig of: : onemix3s/win10 with ubuntu 20.04 vm (opengles2 with vm video hw acceleration off). : ... just to say, it's doable - and probably fairly easily depending on the hardware. - Windows tested some more.. It kinda works now. Still working through tests. - FreeBSD still untested. - Alpine/AWS Linux 1/2: - works (with warning) for glibc build. - works fully with Alpine build. (i.e. no lib-ui support compiled in) [./build alpine upx] - Fedora: works fine. (v33+) - Ubuntu: works fine. (v18+) Operating with the -u startup arg will only have any effect on builds with lib-ui compiled in. (i.e. !noui tag) * added some more types to VAR. (and INIT functionality merged into VAR) * added support for an identifier list in VAR: eg var a,b,c int = 42 * added support for [] and [size] as prefix to type in VAR. library features ---------------- Wrapper around Go Pixel library. Early stages, experimental. The Pixel library gives us a massive amount of extra capability, but does come with some performance issues around threading and supported platforms. Eventually the hope is that it will all be switchable on each platform with the -u flag. * added UI lib calls, so far: - ui_init, ui_close, - ui_closed (has the close button been pressed?) - ui_handle (returns internal window id) - ui_clear, ui_update - ui_title, ui_text - ui_new_draw (create a draw object) - ui_draw_reset (clear object content) - ui_batch (send draw object to update batch) - ui_batch_clear (reset a draw object) - ui_batch_draw - ui_pp (create a point) - ui_polygon (connect prior points), ui_circle (connect prior points) - ui_rectangle, ui_circle_arc, ui_line, ui_bounds - ui_colour, ui_set_smooth, pic_load, pic_bounds, ui_centre - ui_new_sprite, ui_sprite_draw - ui_new_matrix, ui_mat_move, ui_mat_rotate, ui_new_vector - ui_get_code (find a button (keyboard/mouse) depression constant) - ui_just_released, ui_just_pressed, ui_pressed - ui_get_monitors, ui_primary_monitor, ui_set_full_screen, ui_windowed - ui_w and ui_h * other calls are in progress - mainly will be wrappers around basic Pixel functionality so that we have at least some support for windowed applications. - we'll add a number of UI elements over time. * added -u startup flag to indicate requirement for opengl access/windows. * see eg/ui for example code loop. * early testing of performance indicating reasonable results. - for very low object count windows, frame rates anywhere between 150-1000 - low object count, around 100 fps. - 30-60 fps achievable if the draw count is kept down with batches. - you wouldn't ever want to try writing a game with this language, but it is able to keep up on simple GUIs. other stuff: * added more strictness checking of types in almost all existing library calls. - we have added some more generalised parameter checking for types in the standard library. Where varargs are expected it is more relaxed. * added randf() : random float between 0 and 1. (or between 0 and n if n supplied as argument) * added check in VAR to explicitly prevent redeclaration. 1.0.13 ====== fixes ----- * changed deferred unlocks around variable reads to immediate for small speed improvement. * added function concurrency counter to lower lock requirements * inlined vgeti: it's messy, but faster. still just polishing a turd though. * changed statement in Call() to a pointer to Token inside the phrase. - dropped from around 400ms in profiles over 8 secs to ~100ms. * added check for uninitialised variable use as identifiers in eval.go (fn identifier()) - an exception to this is module and enum names that have been created during execution. * updated a couple of eg/ examples to reflect the uninitialised var change above. - ( mainly initialising lists which are appended to ) * added "t" (translate) mode for tr() call. it is still nowhere near as useful as the full tr tool, but it covers most basic requirements. still could do with a negation and upper/lower option. * deprecated optional mutex locking. * locks are now always enforced by default except in a few safe areas. * the locks() library call and the -l startup option will be removed in 1.0.14 * this has obviously had a negative effect on the language speed, but it is a fairly minor reduction in most cases. (<1%). It might get worse in hot code depending on the nature of operations, but it was worth the trade-off for operational safety. * changed system() and Copper() calls to cause assignation of struct instead of string when using the =| operator. This allows for an occasionally more readable use of =|. - Returned struct fields are .out (stdout), .err (stderr), .code (command status code) and .okay (summarised status). - The last() and last_out() lib calls still work as before should you need them. * added default behaviour in interactive mode to print the result of submitted expressions - this only happens when a) not an assignment, and b) no evaluation error occurred. * added support for cursor keys in keypress(). left, right, down, up return 8, 9, 10 and 11 respectively. * lexer - forced numbers to end when followed by 2 dots * made WHEN condition optional ("when a" above could be "when"). the constant "true" is used when the condition is missing. language features ----------------- * added operator <- - usage: <- file - reads file content as a string literal - e.g. on <- "text_file1" == <- "text_file2" do print "same" - added support for use in FOREACH * added description hint option in INPUT ... PARAM statement for error messages. (param #4) - e.g. > input mq param 1 "Maximum queue length" * added (very) simple max column wrap for panes. * added statement ENUM - enumerations can be defined in this way: > enum colours ( red=1, green=2, blue=4, mask=7 ) - for numeric constant type values, when missing, they assume the previous value + 1 - the initial default value is int(1) - string literals may also be used as values. - strings and numeric types may be mixed within the same enum declaration. - further examples, > enum authors ( bill="William Shakespeare", george="George Orwell" ) > enum bitpos ( l0=1,l1=2,l2=4,l3=8,h0=16,h1=32 ) > enum rainbow ( red, orange, yellow, green, blue, indigo, violet ) > enum mixed ( one=1, two=2.0, three, pi=3.141, four=4, author="Orwell") - the final mixed enum has these values: [ 1, 2.0, 3.0, 3.141, 4, "Orwell" ] - enum addressing: - an enumerated label may be dereferenced using the form: > enum_name.enum_label # example: > colours.blue > 4 * added HAS expression condition type to WHEN...ENDWHEN - "HAS expression" clause causes the initial WHEN condition to be ignored and a clause truth test is performed against the presented HAS expression instead. - on falsehood, subsequent clauses are tested with the original condition, except in the further HAS clauses of course. - as before, WHEN clauses continue to match on a first come-first served basis. - context help and manual still need updating for new clause type. - the HAS clause essentially acts as our if..else if..else if..end method, but it also allows for some other alternative arrangements in conjunction with IS, CONTAINS and OR clauses. - the HAS keyword name may yet change for an alternative if we can think of anything suitable. Things like SATISFIES, ADMITS, MEETS, CONCURS and a few other alternative were just too wordy... and yes, I know we have CONTAINS, but that one is up for barter too. * added ternary (query) operator: - syntax: ?? expr true_expr false_expr - i.e.: ?? (ternary if) expr (then) true_expr (else) false_expr - see eg/nestdepth for example in near functional style * added s..e operator for generating an array range. value types of expressions s and e must match each other, and the generated range is of the same value type. - s must be lower than e for all int types - accepted types are currently int, int64 and uint. - it would probably be a fairly simple task to extend this scheme. * forced, combined example: a=rand(50) when a is 10 println "a is 10" has a in 1..9 println "a is less than 10" has a>10 println "a is between 11 and 50" or println "some other value" endwhen * added basic filter and map operators: - filter is ?> and map is -> - both accept a list on the l.h.s and a string on the r.h.s - both accept a # char in the string as a substitute for each l.h.s value - both produce a list of the same type as input - both accept lists of bool, uint, int or float - filter has a slightly higher precedence than map - reduction will probably not be implemented as it can be conveyed with a user-defined function or library call as with max() and avg() below. - may be worth considering a way of auto-promoting larger filter and map operations to parallel tasking by splitting the l.h.s. - would require some kind of local/remote pipeline and some resiliency on failed transfers and result gens. - this is pretty far out of scope for the language so it will probably gather dust. - update: added support for arrays and maps of simple interface{} types. - this means that array literals should now be supported too. - examples: >> print 1..20 [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20] >> print 1..20 ?> "#>5 and #<15" [6 7 8 9 10 11 12 13 14] >> print 1..20 ?> "#>5 and #<15" -> "#*3" [18 21 24 27 30 33 36 39 42] >> print ( 1..20 ?> "#>5 and #<15" -> "#*3" ) ?> "#>29" [30 33 36 39 42] >> print ( (1..20 ?> "#>5 and #<15" -> "#*3") ?> "#>29") . avg 36 >> print ( (1..20 ?> "#>5 and #<15" -> "#*3") ?> "#>29") . max 42 >> print list_float(70..90) -> "deg2rad(#).sin" [0.9396926207859084 0.9455185755993167 0.9510565162951535 0.9563047559630354 0.9612616959383188 0.9659258262890683 0.9702957262759966 0.9743700647852352 0.9781476007338056 0.981627183447664 0.9848077530122081 0.9876883405951377 0.9902680687415703 0.9925461516413221 0.9945218953682734 0.9961946980917454 0.9975640502598242 0.9986295347545738 0.9993908270190958 0.9998476951563913 1] * you may also filter against simple assoc array types with string keys and value types of bool, int, uint, float and string. see eg/op_filter for basic example of assoc array filter and map. PLEASE NOTE: These operators are very naive. They will consume a lot of memory if you let them. They do not do anything clever with list manipulation, generators, yielding, comprehension or any other single thing that would help lower resource cost. They just brute force over the lists. This may improve in the future if heavily used. Likely not though. For now you can expect a very long list of numbers (e.g. 100 million) to take a minute or two to filter or map. 1 Million items will probably take a couple of seconds. We will add support for []struct and map[string]struct later if required. (# would probably have to stand in for the variable name in the case of arrays. i.e. new_array = old_array ?> `#.name=="smith"`; or something similar. not sure about syntax for map[string]struct yet, but would likely be the same! ) * statement line calc (again!) (lex.go + actor.go) - improved again - remains to be seen if it is actually correct now. needs more test data. - disabled SourceStore allocation and use - shouldn't be needed now. - just using the .Original text from phrase for reporting now. - i think the line calc in TEST or ASSERT may still need adjusting. will fix in 1.0.14. * release_id() on windows now returns "windows" library features ---------------- * added library calls: - enum_names("enum_name") : returns a list of labels for a given enumeration - enum_all("enum_name") : returns a list of values for a given enumeration * added library call permit("flag_name",bool) to amend strictness of some run-time behaviours: - permit("uninit",bool) : uninitialised variable use warning. - permit("dupmod",bool) : duplicate module import warning. - permit("exitquiet",bool) : smaller exit error messages * added capture_shell(bool) call. True (default) ensures output of commands in parent shell are captured. - false will disable stdout capture so that commands such as ssh will operate correctly. - this is currently limited to parent shell { coproc(false) } activity. - in the normal child shell output will always be captured. * added json_query() call. this is just a wrapper around gojq for simple queries: https://github.com/itchyny/gojq - pretty great software - saved me a ton of work! :) - will have to see what impact this has on binary size, performance, etc before deciding if we should just generate a more basic version of our own. I really would like to keep this package in place though. it's going to help a lot when using this language with cloud-based tooling (aws and similar). * added btoi() and itob() lib calls for converting bool to/from int. * added zip() and scan_left() list library functions. - zip accepts arrays of bool, uint, int, float64, string and mixed (interface type) - scan_left (currently) accepts arrays of int and float64 - zip will interleave the elements of two arrays - scan_left performs a cumulative operation against each element of an array, with the resulting value pushed into a new array. a seed value can be set to use against the head element. 1.0.12 new model for identifier binding. * finally replaced the identifier lookups with something approaching a name binding scheme. the language should be somewhat faster now. - BREAKING CHANGE: as part of this rework we have had to remove support for l.h.s. variable interpolation. it may be re-introduced later if necessary, but it will always suffer from slower relative performance due to its nature. - All functions, including global/main are now pre-parsed on first call to tag identifiers with allocation slots in local storage. - Where evaluation is, by necessity, more dynamic, lookups are still performed instead. * some example code removed that required l.h.s. interpolation. * modified evaluator to not throw an immediate error on illegal array access. - i.e. variable not found in v[e]. - this allows for comparing v[e]==nil (or nil==v[e]). - this results in quicker empty element checks than !key("v","e") - may change back yet, haven't figure out all the downsides yet. * added and, or and not as synonyms for &&, || and ! 1.0.11 SUMMARY multiple return value handling simple in-expression assignment result chaining multi-line REPL input compound assignments and sqr,sqrt operators pre/post increment operators split interpolation into local replacement and evaluation types. DETAILS * breaking change - renamed start() and end() to hasstart() and hasend() * breaking change - adjusted some functions to return structs instead: - dir() tokens() zainfo() web_get() * breaking change - adjusted fields() to only return an array (F[]) - it was bogging down the var count per function when you used fields() on a string with a large number of fields. * experimental - added partial support for module namespaces. - modules must have filenames exactly matching their namespace name (i.e. no extensions) - namespaces must follow conventional identifier rules (no hyphens, start with letter, etc) - on function call, a reference such as namespace.funcname(q) will call function "funcname" from module file "namespace" with args "q". - without a namespace and dot a plain funcname should call the last defined version that it can find. (defined during execution of the current function space) - no idea if this will work properly yet, but shouldn't interfere with current naming. - see eg/func.namespaces for example. - added a filter for module filenames ending in .mod. these should work as well as module names without extensions now. - no plans to extend this into variables/struct at all. just a convenience facility. - no plans for module aliases. this point may change though. * added regex operators: ~ : direct match (result:bool) e.g. "abcd" ~ "d" => true ~i : insensitive match (result:bool) e.g. "abCd" ~i "c" => true ~f : filtering match (result:[]string) - e.g. "abcd" ~f "." => [ "a", "b", "c", "d" ] - e.g. "abcd" ~f ".{2}" => [ "ab", "cd" ] - e.g. in iteration: >> foreach c in "abcdef" ~f "." -- println "__",c,"__" -- endfor __a__ __b__ __c__ __d__ __e__ __f__ >> foreach c in "abcdef" ~f ".{2}" -- println "__",c,"__" -- endfor __ab__ __cd__ __ef__ * added str * pos_int : repeat string operator * added []string and []uint to + operator. * added IN operator for arrays: a in b returns true if value a present in ary b - can be negated thus: ! a in b - operands bind less tightly to logical negation than IN. * added simple scalar assignment in r.h.s. expressions. - the value of the expression is the same as without the assignment involved. - only simple direct assignment to locals is supported. - i.e. print a=10 # this is fine - print a[0]=10 # not good - print g=10 # not good if g is a global - extended compound ops (++,--,+=,-=,*=,/=,%=) to work with globals - setglob now supports all of the above compound operators. - still only work for simple operands on l.h.s. * comma operator: tldr; there isn't one - behaviour is, um, sporadic, at best. should try to avoid it currently. - i.e. print b=a,a+=b # this is fine, will work as expected because the statement # processes the commas. b=a,a+=b # only b is assigned here (with initial value of a) # normal parsing of expressions considers a comma an end of expr. # so the a+=b would be ignored. - this currently works: a,b,c=[1,2,3] - ... but this won't: a=1,b,c=[2,3] - this behaviour may change, it's a fairly simple amendment. but not sure that the use cases would justify it yet. * return values: - modified return command to return either a single value or an array of values depending on the argument list provided to it. - amended doAssign() to handle a comma separated list on l.h.s. - rules are currently (but may change yet): - #lhsargs > #rhsargs : syntax error - #lhsargs < #rhsargs : all return values returned to lhs var as an array - otherwise, a direct left-to-right mapping. - handling for multiple return assignment is pretty slow at the moment. still working on improving this, but should be working now. - e.g. define swap(a,b); return b,a; enddef x=10; y=20; x,y=swap(x,y) println x," ",y >> 20 10 - x=swap(x,y) println x >> 20 - x,y,z=swap(x,y) >> Error in evaluation >> not enough values to populate assignment - # with result chaining, mentioned below: x=10 y=20 x,y=x.swap(y) print x 20 print y 10 * added basic multiline entry to repl - checks for a few of the nesting token types and keeps accepting input until they are resolved. - changes to a continuation prompt during extended input * experimental - added result chaining * e.g. a.b.c.d : passes the result of expression 'a' as the first argument to b and so on. * a.b(x): makes a call like b(a,x) * as function arg types are not checked this can cause silent failures currently. * feature may be removed or amended at short notice before final release. * this feature does not currently work *directly* with literals. * in order to use with numeric literals they must be parenthesised, e.g. - define double(x); return x+x; enddef - print (3).double > 6 - print ("test").double > testtest - print "test".double > testtest * if a.b is a field reference 'b' of struct 'a' then it will take priority over chaining. * added +=, -=, *=, /= and %= for local and global scalars. - (experimental) added support for array elements and struct field references on the l.h.s. of the compound operators above. - also added for -- and ++. in expressions pre- and post-inc/dec are still supported. - The SETGLOB statement must be used for global scope assignments * PLEASE NOTE! these operations are in fact slower than their long-hand equivalents. a copy of the whole expression is made and then manipulated in order to enable them. As the memory for the copy is allocated dynamically you should avoid these operations where possible, if you have any expectations of performance. - these operations are in place mainly for expressiveness in code, not for performance. - this may improve in the future, but for now, avoid where it matters. * added sqr and sqrt as unary operators * added field deref on array elements (lhs+rhs), i.e. a[i].f=a[j].f * will only support one level of this (i.e. no multiple [] or . on lhs) * removed deval() test function. * breaking change - separated interpolation into { ... } for var substitution and {= ... } for evaluation. * if you need to read a global variable in substitution now, you should eval {=... } instead. ( or read the variable locally and substitute as normal ) * the reason for the change was to permit such things as =| awk '{print $0}' to work without the evaluator thinking you were trying to use a statement (PRINT) as an identifier during eval. * you could also have turned interpolation off temporarily around the command, but this then meant you could not perform a secondary sub in the same command, e.g. a =| awk '{print $0}' <{file} # {file} would be ignored. * breaking change - removed ZERO, INC and DEC statements. * these were becoming more troublesome than they were worth to maintain. * ZERO was already effectively replaced with the more flexible VAR statement. * commands replaced in eg/ code. * added pre- and post-fix decrement and increment for numeric types ( (u)int(64),float64,uint8 ) * needs more testing * added a bodge to the parser to keep hold of the previous token ( added another one for the pre-previous token too, for the compound assignment ops ) - not how it's supposed to be done, hence more testing. - we only keep the accumulated left-hand value not the token normally. - bodge should allow us to add -=, +=, /=, *= and %= fairly easily now. * fixed tail-call elimination (hopefully!) after the changes for multiple return values broke it. needs some more testing, but appears to be correct. 1.0.10 new evaluator, bug fixes and documentation updates * New evaluator written for some small speed up and better integration with whole. * features will be detailed below. It is about 95% feature complete, I think. * existing programs should run fine, but there may be a few details I've overlooked... * re-uses existing token streams (except in interpolated sub expressions) * pre-1.0.10 we were re-tokenising from input string (mostly changed now to token streams) * error capture - need to pass these back up to .Eval()/deval() uniformly * done quite a lot of this - maybe a few small things outstanding. * new evaluations from a given functionspace should re-use a parser instantiated at the start of the function Call(). * the parser is passed through sub eval routines which require it. * this saves on initialising the led/nud map repeatedly in many cases. * will probably still happen in recursion, * definitely will happen in interpolation and user eval() calls. * need to work around the interpolation one definitely. i'm probably missing a trick here. * DONE, maybe? Added a separate parser instance just for the interpolation routine, and threw a mutex around it. should be good enough. * initially only supported basic arithmetic (-+*/%). But, we have now added: * float support * equality operators (==, !=, >, <, <=, >=) * float comparison gets fuzzy around the 16th decimal place * power (**) operator. * logical operators: negation ! , bool && and || * decimal shifts: shift left and right operators: << >> * string literal support * array literals: [ x,y,z ] * identifiers: needs more testing but pulling sane values and types from scope so far. * identifiers cannot be named the same as reserved keywords (and it doesn't crash now!) * identifiers can share names with library functions or user-defined functions. * bit-wise operators: ^ (xor), & (and) and | (or) * booleans: added true and false constants. some testing done. seems okay. * user-defined function integration * standard library integration * for both of these, there still may be some clean up to do in order to reduce the number of chained function calls these pass through. (ev/udf/Call/interpolate/buildRhs/etc) * getglob() is now optional! * globals are now checked for a match *after* local var names have been checked. * The SETGLOB keyword is still mandatory. (and will probably remain so, and not for technical reasons) * performance: the new evaluator started with terrible performance, as most things I write tend to do :) * the old evaluator (1.0.9 and below) was around 12-13 times slower than python on same machine * the new evaluator started life around 4-5 times slower than python. * we are now at around half the speed of python evaluation after a little tweaking. * we will continue to tinker with this between releases, but expect it to get steadily faster. * i will run out of ideas eventually (probably not far from now) on how to improve it, but the basic issues are avoiding mem allocations, avoiding defers, avoiding mutexes (to a small degree), avoiding maps, using smaller datatypes, avoiding even array lookups, cutting out function call chains, inlining more of the smaller things, forcing things on to the internal stack, etc, etc. * better integration of pre-1.0.10 features should also improve speed. * i'll keep chipping away at it :) ************************************************************ * currently best performance comes from using fixed arrays * * where you can (with INIT) and avoiding interpolation. * ************************************************************ associative array performance is acceptable, but things like map writes are still about half the speed of python. * for tight loops that only require a simple addition / subtraction it is still much faster to use the INC/DEC statements. * INC and DEC are slightly faster than python in the same tests (eg/addition_loop and eg/alt_lang/addition_loop.py) * in all honesty, further performance gains are entirely optional at this point, and quite unnecessary. We'll do what we can with little effort involved, but it is now quite fast enough for the intended use cases, i.e. shuffling data around in simple scripts that replace bash/perl/awk monstrosities. * Performance difference between 1.0.9 and 1.0.10: Measures below from FreeBSD. Ubuntu has similar gains. Windows/Alpine/others not checked yet. v1.0.9 v1.0.10 eg/long_loop 1.71s 1.2s ( 100 million iterations, avg 3 runs ) eg/addition_loop 61.0s 15.1s ( 40 million iterations, avg 3 runs ) eg/map_writes 55s 11s ( 20 million iterations, avg 3 runs ) * Added language version comparison in REQUIRE command. usage: REQUIRE x.y.z Compares the required semantic version to the version of the interpreter currently executing. Bails out with an error if x.y.z is greater than the executing version. * Added command VAR: (experimental) * var is used for declaring an "expected" type. It is not enforced completely, it is more for additional sanity checking. * as an example: var i bool # i=false var j int # j=0 var k string # k="" var l float # l=0.0 var m uint # m=0 i=42 ... will produce a run-time error at "i=42" and execution will stop with an appropriate error. * the underlying storage type will be set to the appropriate type (instead of untyped) and the internal vset() function enforces it. * if you try to VAR an existing variable name in the same function scope you will also get a run-time error. E.g.: i=42 var i int # or whatever else # results in error * permitted expected types are: int (64 bit), int64, bool, float (64 bit), string, uint (64 bit) * you can still continue to use untyped variables without the declaration as before. * this is not meant to be a full type system, it is only intended to help with basic checking. the scope may be extended further to help in other areas, but most likely you'll never use this :) * we'll expand this into compound or other data types later if there's justification to do so, I'm just not sure how useful it will be yet. * another reason for this is that some library functions may be able to perform better if they understand the argument types ahead of time. 1.0.9 * BSD preliminary build: * cloned console_*, winch_* and lib-os_* to freebsd versions. * amended console_freebsd with TCGETS/TCSETS replacement values: TIOCGETA+TIOCSETA * amended build and build module with freebsd specific options (removed upx) * THIS IS A TEST BUILD. IT RUNS, BUT A FAIR NUMBER OF THINGS MAY STILL BROKEN. * eg/flood displayed zero hitching due to the OS. performs better than linux in this regard. * moved TC[SG]ETS and TIOC[SG]ETA to separate const files: : console_const_linux.go+console_const_freebsd.go : moved console_linux.go to console_unix.go : removed console_freebsd.go * added -Q option for reporting specified coproc/shell command options. * fix shell detection on startup for freebsd: anomaly - was missing a @last default set in Copper()/console_common.go * test web server functionality in freebsd: * was fine with some changes to the eg/web example for static hostname. * check concurrency is working as expected for dynamic pages in web server (in freebsd): + siege : siege -b -c 100 -r 100 -k http://127.0.0.1:8080/fun/awsip * siege worked fine for lower numbers than above. would have been fine with higher except that the example page has a link to unpkg.com for pure CSS and this and DNS lookups hit rate limitations upstream. Lower numbers for -c and -r were fine (once throttles dropped away between tests) * siege was perfect with the above (or higher) -c and -r when fetching the static example @ http://127.0.0.1:8080/exmachina/index.html. * because of the changes in this version, general release will be delayed for more clean- up and testing. i felt bsd support + structs were important enough to allow this. * added sequential/random access filing functions: * new funcs: fopen, fclose, fseek, feof, fread, fwrite * see the library html pages for details. * eval lib updated (accessField processing and add/sum/mul/div/unaryminus/mod to handle other signed int types. * added unary positive and improved lexing a little on scientific notation NumericLiteral cases. * removed a few more allocations in FOR..ENDFOR. * replaced a vset() call for direct var access when thread safety not required, in ENDFOR. * started work on STRUCT struct_name...ENDSTRUCT and INIT var struct_name * syntax in place. * added STRUCT and ENDSTRUCT to vim syntax file and online doc. * struct builder done. (actor.go, added struct, endstruct and structmode) * added SHOWSTRUCT [filter] command for listing structmaps. * INIT changes done. (init var_name struct_type) * struct comparisons: * == and != should be fine for shallow comparisons. * as we don't allow nesting, this should be enough for now. * >, <, >=, <= make no sense between structs and therefore generate an error. * eval.go/dotted assignment changes done. * added a check for field name's existence pre-assign. * added a type check for assignability. * added more error handling around various outcomes of struct field assignment. | | example: | | struct s_person | name string | age int | dob string | endstruct | | init person s_person | | person.name="billy bramble" | | define showBirthOf(p) | print p.dob | enddef | showBirthOf(person) | | etc, etc.... * struct definitions are available throughout all scopes. * struct instantiations are locally scoped. * only some basic types are supported for fields: * bool, int, float64, string * int32, int64 and uint64 should also be supported, but need more testing. * i am very unlikely to add more complex types as field types (arrays/maps/etc). * the intention here is for structs to be used for bundling config and improving stdlib options. * for this, we do not need anything more complicated. will see how it goes though... * the structs are formed with compatibility in mind to the underlying Go language. * this will allow for their use in the Za stdlib and a few other hard to reach places. * unlike Go, there is no enforcement of private/public capitalisation of field names. * that is, all fields are available regardless of casing, within Za. * structs should also allow for some slightly cleaner passing around of bunches of config. * serialise and deserialise struct to string/json/something for save/load: * new lib calls: write_struct(file,source_struct_name) and read_struct(file,dest_struct_name) * these functions take a flat struct variable and send them to or from disk. * read_struct requires a reference struct as a template for populating the return value. * the usual process would be to define a struct, initialise a var as that struct type then use the empty initialised var as the destination struct which read_struct will populate after examining the layout. * some vague examples here: eg/struct, eg/struct[23], eg/struct-serial !! !! !! there are quite likely a ton of bugs with struct handling currently !! !! USE WITH CAUTION !! !! !! * STRUCT parsing limitations: * dereferencing: array of struct * it is possible to declare an array as mixed type, eg: | init a mixed 10 * and to then assign a struct value into an array element: | init b s_person | b.name="test"; b.age=10 | a[0]=b | print kind(a[0]) > struct { name string; age int; dob string } * if you assign a 'blank' (initialised, but no fields assigned to) struct value to an array element, then unspecified behaviour occurs. generally, the array element will assume the type and zero value of one of the struct fields of the whole struct assigned to it. populate the struct before assignment to the array element! (at least partially) * you can access fields directly, eg: | print a[0].name * or retrieve an entire struct record: | c=a[0] * it is currently not possible to mix dotting and arrays in assignments. * to operate on elements of a field, you must pass the object or the field through temporary vars, eg: | c=a[0] | c.name="new_name" | d=c.name | print d[2] # 'w' | a[0]=c | print a[0].name # 'new_name' * you cannot directly access elements of a dotted array element, eg: | print c.name[0] # error | print a[0].name[0] # error * as usual, I cannot improve much upon this without rewriting the parser and evaluator "properly". * the situation is similar for associative arrays: | struct S; a int; b bool; endstruct ; showstruct > S a int b bool | init a S | m["first"]=a | print kind(m["first"]) > struct { a int; b bool } | for e=1 to 5; m[rand(10000)]=a; endfor; print m > map[1319:{0 false} 1848:{0 false} 2082:{0 false} 2541:{0 false} 3301:{0 false}] | print m[1848].a > 0 | print m[1848].b > false * added write_struct and read_struct lib calls. (to/from disk) * added uint lib call (for type conversion) * more doc cleanup * moved interface{} type cases to end of switches. * fixed error in reverse() lib call due to the new placements. (type assertion error revealed in previously uncalled code). * fixed a bad append() in eg/mon (int to []float) * console_windows consolidation review * split common functionality with console_linux to console_common.go * still a couple of other funcs that could be broken up, but okay with those for now. * added terminal object nil check in term_complete() * added expand_aliases and -i to bash startups. it still won't pull a startup file in, but at least the alias command will work again for new aliases. * fixed bug in non-shell interactive mode exec calls. * added existence and empty checks in INPUT id ENV var command. * the Za id variable is left intact when env var is empty. * if Za id variable didn't exist then it is created regardless of env var existence. --------------------------------------------------------------------------------------------- 1.0.8 * added ctrl-c handler in keypress() func. sig_int is now raised when detected, but keypress still returns as normal. this allows a user interrupt handler to still be defined, or for the interruption to be ignored and the ctrl-c (dec:3) to be captured in keypress(). * after testing with sh/dash/ash and similar, found the echo -e required in coproc calls was problematic. (didn't interpret escape codes too well). * added check for custom shell basename of sh, ash or dash. when this occurs, then external printf command is used instead (/bin/printf). * will try and come up with a better workaround later. * no extra system variables output that are sh/ash/dash specific yet. * zsh seems fine still, except for only putting out a single zsh specific sysvar. will work on more of them later. * will deal with WSL/mingw/windows'y things later. * fixed repeated enter key glitches in interactive mode. (was more slow ANSI VT response issues). * added check for terminal availability and ready-to-send status in GetCursorPos(). * mainly added for docker compatibility when not using -it in exec/run. * added some more functionality around handling "all interfaces" 0.0.0.0 web service requests. * http.ListenAndServe now split in half and tcp4 request servicing enforced. * removed the enforced 404 return when hostname/ip not found in web rules * instead a secondary check is done when the request host name not found for rules which contain 0.0.0.0:reqPort instead. * these changes were done to allow the web server to be used more successfully from inside docker containers. * added getcores() to retrieve cpu core count. --------------------------------------------------------------------------------------------- 1.0.7 * console: key repeat was not behaving. holding down key caused glitches. * GetCursorPos() call (ANSI VT code call+response) was a little slow anyway and opening and closing tty constantly was getting to be bad news. * now keeping a handle throughout execution for dev/tty * update: now tested, seems okay. *tentatively backs away slowly from the scene of the crime.* * still a small issue with repeated ENTER presses. cursor is jumping about. nothing urgent to fix though - cosmetic. * added date functions: * time_hours,time_minutes,time_seconds,time_nanos,time_dow,time_dom,time_month,time_year * interactive mode: * cosmetic fixes in console_linux.go/getInput(). stopped colour codes and percent signs wreaking havoc. * some more examples in function docs. * renamed deq to pop * renamed push to push_front * append() already does what push(_back) should have * added peek() -> returns ary[len(ary-1] * libdoc: updated stdlib help descriptions some more. still some way to go on this particular labour of hercules. * added stdlib tests: db, net * added -U startup option this option is for specifying the byte to be used for signalling end-of-output of a co-process command. By default, this is 0x1e (31) If this may clash with something in your environment, you should specify an alternative with the -U option. --------------------------------------------------------------------------------------------- 1.0.6 * clean up of source layout and use of modules. * if compiling from Go version 1.11/1.12 then source GO script before first build. * this excludes changing the goval code, it's far too entwined now. * added go.mod/.sum for other modules, internal archiving updated to exclude the cache * will monitor for problems, before opening up the github repository. * will also probably need a rebase/cleanup of old version code. * added: int=next_match(s,regex,start_line) # to return matching line number (0 based) * interactive mode * added a little more sanity to multiline input on bottom terminal window line and at line endings. * fairly sure it will still be broken in a multitude of ways. ( such as navigating multiline history ) --------------------------------------------------------------------------------------------- 1.0.5 features: * added stripansi() lib call. * added addansi() lib call. * added uninstall() lib call. *experimental, not tested in all OS yet* * added download() lib call in lib-net. Like web_download() but auto names and provides console feedback. * added func_categories, func_inputs, func_outputs, func_descriptions lib calls. * these are to support doc generation. probably no use for anything else. fixes: * added an inline'able strcmp (internal fn) to use in hot spots instead of built-in string equality. this speeds up inc/dec and interpolation somewhat. * added up-scaler to loop allocator * fixed issue with calltable id allocator not reserving instances before async tasks swamped it. new allocation requests now have faked allocation slot info on exit from GetNextFnSpace(). * amended calltable id allocator to loop while req_ifs>cap as async allocations were busting it a new asshole. (instead of only allocating once) * added deallocator to calltable[] actor.go/GetNextFnSpace(). * added deallocator to loops[] in actor.go/Call() * updated help descriptions for library calls. * updated arg names in library call help. * added check on user-define funcs for names clashing with stdlib funcs misc: * added eg/tests/libcov for vaguely checking library test coverage. * function reference * auto-generating now with libdoc tool. still needs some work. * za-ref.html now contains a link back to zalang.org/funpages/index.html instead of maintaining two lots of fn descs. * libdoc is executing during "docpull" run. * example pages can be created by hand and placed in a prepared location for inclusion in the final page output. --------------------------------------------------------------------------------------------- 1.0.4 fixes: * found another read lock race cond in lib-net. fixed. * updated source comments * updated error reports and help texts * removed more dead code * removed a Call() parameter left over from separate call modes. * this should speed up recursion considerably. It was passing an empty struct in. * it's still slow, just not as much :) * changed actor.go/inbound to use pointers instead of copying phrase from source. * again, there's another small speed bump in places due to this. * re-enabled some eval.go/ev() error capture. * expanded sort() functionality a little * added a few more of the missing iteration types in FOREACH * added argument 2 in funcs() to return (true) or display (false:default) the results. * updated help text * bah, had to tinker more with interpolation. was throwing errors in the background when it should have been ignoring unevaluable contents of curly braces. * added shouldError param to ev() and interpolate() @note-to-self: don't go back in that code for a few weeks, give yourself a chance to heal. * updated (all?) functions in lib-list and lib-string to check OS and determine correct EOL * this is mainly for splitting and rejoining strings/arrays correctly on both Linux and Windows * chances are I missed a few :) linux has been tested, but not exhaustively on windows yet. * updated build script (cosmetic) * changed calltable to scale up dynamically * fixed remaining issue in fields/field calls for CRLF * updated eg/mon. * changed call format and allocation method for new functions then had another disagreement with concurrency. * fixed the leaky dinghy and all seems well again when testing web servers. * added some more error checking in lib-string (trim, start and end) * more bug squishing in lib-string. * some more fiddling with lex.go after changes to eval.go, brace nesting and array assignment. * changed keypress() to encode £ symbols as decimal 163. although we don't support utf8 input or extended ascii in any meaningful way, we still needed to distinguish £ from # on key capture. * updated za.vim for WITH...ENDWITH and tco+echo+unmap+getcol+getrow * added capture of current text cursor position on startup in non-interactive mode. this was to ensure the PROMPT command uses the last cursor position *or* a position set by AT statement. * updated key and globkey to avoid a type exception in arguments. * added search-by-category back to the funcs() call. (as well as by function name). * added pass through of the current function id to the expression evaluator to get rid of another global that was always screwing with multi-threading (lastfs var). features: * modified the build script to produce a cut-down windows version * ansi colour support is automatically disabled on windows * it can be re-enabled with either -C flag or ansi(true) at run-time * the interactive mode is probably a bit crippled * getch(), wrappedGetch() have been reworked, but interactive mode will only work if you use a VT-capable terminal, such as ConEmu. * added a timeout capability to the Windows keypress() version. * the co-process is disabled * console resize signalling (SIGWINCH) is disabled * i don't have any burning ambition to get this working fully, but the executable works given the caveats above. Getting those things working is an incredibly low priority for me... possibly never. * to build: ./build win (generates za.exe, copy it where you want it). * added -C flag (force ANSI colour on) * added ansi(bool) to enable/disable ANSI colours support at run-time. * added separate handler for executing windows commands in | and =| * uses "cmd /c {command}" * may add some support for varying the command interpreter and options later. * added windows detection in col() for CRLF splits. * updated term_h() and term_w() to fetch approximately correct terminal dimensions on windows * possibly depends on the console in use, not tested many yet. * you can now wrap up | and =| command call text in backticks (`cmd`) to allow semi-colons in command. * added system(str[,bool]) for executing a more constructed system call. display output when bool is true. * added dir() lib function. dir(path[,filter]). returns a list of directory entries as an assoc array. * each entry consists of these keys: "name", "size", "mode", "mtime", "is_dir" * added a check in ASSERT to warn if an assignment is present. * For example, the following would cause a fatal exit: ASSERT strpos(t,"we ",0)=0 * added: WITH id AS f ; ... ; ENDWITH construct * !!! may still have some issues !!! * leaking some temp files on exit still * this will be the equivalent of: f=|mktemp; write_file(f,id); ... ; rm -f {f}; unset f * on exit (of any type) it has to clean up the temp file(s). * filename should be available inside ... part. as an example, it will turn this: f=|mktemp write_file(f,s) ac =| awk 'END{print NR}' {f} wc =| wc -l {f} | cut -f1 -d" " | rm -f {f} into this: with s as file ac =| awk 'END{print NR}' {file} wc =| wc -l {file} | cut -f1 -d" " endwith * added: html_escape(s) and html_unescape(s) lib calls. * added: delete(f), rename(sf,df) and copy(sf,df) lib calls. * added: bool(s) for converting string to bool type. * added: umask(int), chroot(string), cwd() and cd(string) lib calls. * was going to add some setsid/setuid/setgid variants but it looks like they are not going to play well with either Go's concurrency or future compatibility as they have been left to rot in the syscall package. Probably just as well - think they may have turned ugly later :) * added: eg/args for processing CLI args. * added: eg/strings tests for string library. * started work on tail call optimization * in the first instance (and probably the only one!) we have added some simple tail recursion elimination on return statements Under these conditions the function will be re-used: 1. executing a RETURN statement, and, 2. there is a function call to the same function name already executing, and 3. the function has parameters, and 4. there are no other subsequent parts to the return expression than the function call. If these conditions are met, then various counters are reset, and each argument in the return expressions are evaluated and their results stored in the variables defined as input parameters for the current function. Subsequently execution jumps back to immediately before the point at which the first line of code of the function is executed. This has had a massive impact on certain forms of recursive function. Please see the examples eg/fib.nomemo and eg/fib.tailrec for comparison. Obviously, you still have to write recursive functions in a manner which takes advantage of this. It's still interpreted, after all! * added: tco() library call, to check if the current function is in a TCE loop. This is useful for keeping counters in benchmarks. * added: echo(bool[,string]) lib func. Enable (true), disable (false) or return the status (no arg) of local console echo. * if second option provided, it sets the character mask for hidden input with PROMPT command. * added: getrow() and getcol() call in library to use GetCursorPos() func. * added: (internal) GetCursorPos() equivalent for Windows. * added: unmap(a,k) to remove a map key 'k' from array 'a' * asynchronous calls: * new statement ASYNC * usage: ASYNC handle_map f(...) [key_name] * ASYNC will run function f() in the background, with given arguments (...) * handle_map is the name you wish to give to ASYNC as a storage list into which a handle is placed for the asynchronous task. * key_name is an override for the handle id in handle-map. * if a handle_id exists in handle_map then you should assume it is still running. * it may have completed, but you won't know until you next poll for results with await() * new standard library call: await() * usage: results=await(handle_map,wait_flag) * await will poll for results returned by tasks launched by ASYNC. * You must provide the handle_map into which the async task was handed-off. * When the function provides a return value, it is placed into results[handle_id] * If a wait_flag is provided, then await() will either wait indefinitely for all functions in handle_map to complete execution (flag:true) or will just poll each in handle_map for completion whilst populating the results list, but return to the caller immediately upon check completion. * as a task completes, await() will remove the handle_id from the handle_map. * example case: # some background task define f(z) pause rand(3000) return z*2 enddef # launch all in parallel for x=0 to 99 async hndmap f(x) x endfor # collect results res=await(hndmap,true) for e=0 to 99 println "{e} -> {res[e]}" assert res[e]/2 == e endfor * if you do not care about the order of the returned results, you can skip adding a key_name in the async statement. * a unique key is generated in these circumstances, i.e. "async_f@39495943999" or similar. * if you plan to use async with functions including shell interaction then you should use the -S option at launch. * attempts to run shell commands in parallel may otherwise return interleaved results and assorted other unpredictable behaviour. * we can probably fix this with a command queue, but there are no plans for that currently. * it would probably also be wise to use the -l option (or the locks(true) call at runtime) to ensure that all mutex locking is enabled, for the same reason. this should not be necessary, but this is a new feature after all! * added: coproc(bool) call. This is to enable processing of | and =| statements using either the co-process shell (true) or the current execution pid's shell (false). This was added so that async command calls could be safely processed. On windows, by default, execution is in the current process rather than the child shell (as we use 'cmd /c' calls currently on windows.) 1.0.3 features: * added json_decode() to convert string to mixed map * added json_format() to indent a json string note: this is only a roughing out of json support. we only really need to be able to read and process json files at a minimum. example use: # get json file str=read_file("eg/test.json") # convert to mixed type map print json_decode(str) map[bugs_collection_link:http://api.launchpad.dev/beta/bugs people_collection_link:http://api.launchpad.dev/beta/people] # format original string print json_format(str) { "people_collection_link": "http:\/\/api.launchpad.dev\/beta\/people", "bugs_collection_link": "http:\/\/api.launchpad.dev\/beta\/bugs" } # pick out value b=json_decode(str) print b["people_collection_link"] http://api.launchpad.dev/beta/people # process foreach e in b print format("%v (%v) %v",key_e,kind(key_e),e) endfor people_collection_link (string) http://api.launchpad.dev/beta/people bugs_collection_link (string) http://api.launchpad.dev/beta/bugs * added (test) support for shell-less executions. if the -s arg is set to /bin/false, then any | or =| commands are routed through the shell of the za instance parent, rather than a child process. last() and last_out() are still supported, but more limited in stderr output. may give this it's own cli opt arg later. N.B. using in this fashion though you lose variable and env persistence between commands. * added -S CLI argument for disabling the co-process shell entirely. * added has_shell() to lib-internal. * added abs() to lib-math. * this gets inaccurate for floats, e.g. > print abs(-2.30000000000000001) 2.3 > print abs(-2.3000000000000001) 2.3000000000000003 * int uses a different calc. float is using the Go math.Abs fn. * added globlen() to lib-internal. * this returns the length of a variable from global scope or -1 on error. * @bug: if an array or map is expressed as, for example, "a[]" instead of "a" then the string length (3 in this case) is returned, but without error. don't do this! :) * this will eventually be treated as an error. * @note: the length of an array is the same as it's currently allocated cap, e.g. > init q int 10 > print len(q) 10 > print q [ 0 0 0 0 0 0 0 0 0 0 ] > q[11]=42 > print len(q) 20 > print q [ 0 0 0 0 0 0 0 0 0 0 42 0 0 0 0 0 0 0 0 0 ] * due to the above, should you need a measurable length array, please use an associative array, e.g. > z[11]=42 > print globlen(z) 1 * this applies to both len() and globlen() functions. * currently debating if this should be adjusted. probably not going to though. it isn't difficult to keep track of what you are using in a pseudo fixed length array. we only allow the automatic length adjustment to save redimensioning in code. if it becomes enough of a problem for people, we can adjust it. * examples changes: * added basic colour test: eg/colours * amended eg/ascii * added eg/json. the script shows depth processing and colouring of json content. * updated za-ref.html->stdlib list fixes: * stripped trailing \n in parse() * moved locking on lastline global into report() function * tweaked error reporting inside ev(). was throwing false errors during interpolation. * stripped out sys var 'path'. not used by anything. * changed sys var 'pwd' to retrieve from /proc link instead of issuing coproc command. * endfor without for/foreach inf. loop : added lastConstruct check at depth[ifs] * when: silent fail without condition : added check+error message * when: can contain code before first clause not fixing. consider it a present :) will pull this if it becomes a problem. * pane: define - error msg cleanup : removed some debug info * exit - invalid expr causes exception : fixed missing 'ef' eval fault on var check in wrappedEval() * return: returning from global/main? not setting exit code added check for functionspace reference and now passing exit code out on exit. * removed some of the mutex locking, re-arranged the rest. still not perfect, but -race not breaking now. tested with eg/web server. only failures due to req. rate, not concurrency failures. * updated actor.go/C_SetGlob to accept `lhs "=" rhs` instead of only `lhs rhs`. memoizing ackermann test was breaking on interpolated elements including a comma between dimensions. this worked fine using other separators. need to update all setglobs and manual to "=" version. * amended dump() to not display read-only system variables * changed some sys vars to read-only naming * added a shell_pid() lib call for getting the pid of the co-process. * fixed initialisation of var in inc/dec for not existing var names. 1.0.2 fixes: 1. added general support for array types in interpolate(). whole arrays and any simple type should now be converted during interpolation. Array elements are still considered out-of-scope for conversion and should continue to be resolved in the usual manner outside of interpolation. 2. added a default conversion in interpolation that also displays the type when caught. 3. removed single trailing newline character from results of line_filter(). (was added by loop in same fn) may need to remove wrapping collapse() calls that previously corrected this. * there is a chance this will break previous functionality, but probably not * * in almost all cases this new line removal will have been taken care of post call. * * it was always an unwanted artifact. * 4. added new function: interpol(bool). This enables or disables string interpolation. 5. added basic handling of array elements and evaluation inside of interpolation phrases. needed to add a conversion of array elements in interpolation. was able to cover calcs and other eval at same time. This kind of thing should now be possible: a="10"; b=20; c["x"]="bang!" println "{a} {rand(20)} {c[`x`]} {b} {pi()+10}" output example: 10 8 bang! 20 13.141592653589793 Use with caution for now - test feature. Mixing ` and " badly will definitely result in unexpected behaviour. It probably has bounds issues too. Still under review! 6. changed ON..DO to re-enter inside function rather than re-call fn in different mode. removed CALL vs ENACT mode. we instead build a phrase and inject it at ondo_reenter. 7. stripped part of the old multi return value code, replaced with simplified version. 8. changed function startups to reuse space for variable sets (functionspaces) or loop counters if previously allocated. this was to speed up recursive calls a bit. took fib(30) down from ~ 1m40s to 0m38s. fib(20) down to below 1s. new eg/fib example shows how to speed it up much more: fibonacci(92) is approx 0.04 seconds memoizing. 9. fixed a broken empty lexed string case in lexer.go... wasn't returning properly. 10. i may have forgotten to add a couple of ampersands here and there.. nowhere important, just on a few of the mutex locks. nothing to see here - move along! 11. retested with -race and pprof. couple of small edge cases when stress testing the web server concurrency. i guess this is to be expected to some degree, but its more stable now. it was already okay. i'll eventually nail them all. 12. updated examples: eg/web. new example: eg/fib (memoizing recursive) recursion is still slow, especially mutual recursion. need to clean up func calling method before this will change. as usual, not a priority. use a real language if you want to do this kind of thing :) 1.0.1 fixes: 1. added type checks to fields 3+4 of fieldsort() 2. added some support for array/map element literals in INC/DEC - needs similar in ZERO. support not yet in for variable/expression in INC/DEC/ZERO of array/map elements. messy, but same can be accomplished with normal assignments for now. definitely no support for interpolation in INC/DEC/ZERO currently. probably best currently to avoid these keywords if speed is not a concern! they are fine for their original purpose of simple counters, but would be best if they went away entirely. a faster evaluator would allow for that. 1.0.0 pre-release version - documentation/file structure changes --------------------------------------------------------------------------------------------- see docs/ for earlier change logs.