Systems, devices, and methods for software development or modification. The disclosed technology relates to transforming interactions with physical blocks by a human developer on an activity surface into computer-understandable digital information or logic for developing or modifying software (e.g., websites or mobile applications) in real-time or near real-time. The physical blocks are representative of software elements used in software development. For example, the structures, colors, shapes or hardness/softness/squeeze/bend/flex/elastic/shape-memory/rigid properties, whether symmetrical or asymmetrical, whether open-shaped or close-shaped of the physical blocks can determine which software elements are being developed and the arrangement of the blocks can be mapped to how the software elements are to be included in the software. Users located remotely from the developer can provide annotations or feedback to the software being developed in real-time.