Closures vs pureness

The other day I was looking at a nested map/reduce/filter constellation which had a bunch of nesting, therefore there were lot of closures. This colleague had an interesting question: "In PHP, usually we can tell the interpreter that a function is relying on something from outside of the function with the `use` keyword, so e.g. we could tell at one level of the nesting that a function not only relying on it's input, but something from the outside (closure). Is there a way to do this in JavaScript?".

Let me show this on an example. (this is a simplified version what you can find at http://reactivex.io/learnrx/ .

Now, are these nested functions pure? No, not really. I couldn't simply unit test them because they need context outside of their closure.

This made me thinking, how could we still achieve pureness?

As anyone, I first tried to Google it (khm DuckDuckGo? Sorry, I'm still not detached from the search giant..). I found this SO thread. The solution seems easy and elegant, how could I not think of that before?

So let's try to refactor our code:

So here you can see that videoBoxartMapper receives 2 arguments, so we can exlicitly depend on the context coming in from the outer function. We could easily achieve that by simply return that function and passing the video argument to it. Tada!

In summary, I think doing these crazy nestings can be harmful from a readability perspective, but as you can see, even this can be refactored in a pure fashion, so yay, we can easily test them, but one needs to have this mindset from day 0 when implementing deeply nested closures.

Observables vs Promises vs Iterators vs Functions

On this page, I found something I was searching for a long time: http://reactivex.io/rxjs/manual/overview.html#observable 

	Single	Multiple
Pull	Function	Iterator
Push	Promise	Observable

This is a summary of how Observables, Promises, Iterators and Functions are connected to each other.

I find it extremely helpful for people who are just getting familiar with Observables in general, because it helps you place the concepts in your current mind map.

So to sum it up it divides the access mechanism of data into 2 categories: Push and Pull. Pull means that the caller explicitly requests some data and awaits for it. This means that the one who's pulling knows when exactly it's going to be pulled (sequentiality). Pushing on the other hand means that one just listens to a certain channel and waits for data to be received. The advantage of that is that the provider of the data knows exactly what it's going to provide to it's listeners.

The other perspective to look at accessing data is how many times it can provide. It can be 1 time only (Single) and more times (Multiple).

These add up to the table you can see above, where every cell represents different concepts you're already familiar with.

It's pretty important to mention that one can jump around between different types - with obvious restrictions. For example one can create an Observable out of a Promise. It will be a specific Observable that it will fire once, and then it completes, but it is possible. One can do the other way around as well, but it's not going to be the exact same as one could expect. Because a Promise is a Single Push operation, this means that even if we cast somehow from an Observable to a Promise, it won't behave the same as an Observable, but will only fire once, as one could expect that from a Promise.

For the record, one can trick the system by creating a Promise that awaits for multiple sources, so in theory could mimic the behaviour of an Observable, but that would be very hacky, I'd not suggest to do that.

The Mythical Man Month

 I am reading currently The Mythical Man-Month because I feel like it's such a valuable resource to summarize my general project management knwoledge in a readable format.

I found this link, which summarizes the whole book pretty well, I wanted to save it for my personal collection, so I'll just copy it over to find it in the future (been searching for it lot of times, got tired of it haha).

Tar Pit (What is programming)

Software like a tar pit: The more you fight it, the deeper you sink!

No one single thing seems that difficult; "any particular paw can be pulled away." Simultaneous and interacting factors brings productivity to a halt.

A program is just a set of instructions that seems to do what you want. All programmers say "Oh, I can easily beat the 10 lines / day cited by industrial programmers." They are talking about just coding something, not building a product.

A product (more useful than a program):

• can be run, tested, repaired by anyone
• usable in many environments on many sets of data.
• must be tested
• documentation

Brooks estimates a 3x cost increase for this.

To be a component in a programming system (collection of interacting programs like an OS):

• input and output must conform in syntax, semantics to defined interfaces
• must operate within resource budget
• must be tested with other components to check integration (very expensive since interactions grows exponentially in n).

Brooks estimates that this too costs 3x.

A combined programming system product is 9x more costly than a program.

Why is programming fun?

• Joy of creation
• Fun making things people use
• fascination with complex puzzles
• curiosity, learning
• working in an etherial medium like a poet

On the other hand...

• you have to be very precise
• you rarely control what you are to build (specs)
• dependency on other programs, programmers
• finding bugs is a drag and very difficult
• takes forever to build anything big
• product obsolete when you finish; people are after next big thing before you finish (can make the current project wander and never finish)

Mythical Man Month

Why does software fail?

• Difficult to estimate time
• Optimism
• Effort != progress
• Schedule not monitored
• Slippage induces addition of people

Brooks says programmers are optimists (everything will go right etc...). Incompleteness and inconsistencies become clear only during implementation. He concludes that experimenting, "working out" are essential disciplines.

Each task has a nonzero probability of failure or slippage. Probability that all will go well is near zero.

Cost varies with manpower and resources, but progress does not! Hence, using "man month" (person month) as a measure is misleading and dangerous. They are interchangeable only when there is no interaction whatsoever between tasks.

• Draw line downwards top left to lower right months (y) vs people (x) for a perfectly partionable task.
• For semi-partionable task, show convex downward curve top left to bottom right
• For unpartionable task, show flat line from same months vs people graph.

For partionable tasks that require communication, must add communication to the cost of completion. Communication is:

• training (technology, goals, overall strategy, plan of work)
• intercommunication; if each pair must communicate, cost grows with "n(n-1)/2". Meetings of >2 people makes it even worse!

For building a system (requires lots of communication), the communication effort quickly dominates the effort. Adding more people lengthens not shortens the schedule

Testing cost underestimated always. Brooks suggests:

• 1/3 planning (yes this costs a lot)
• 1/6 coding
• 1/4 component test, early system test
• 1/4 system test, all components in hand

TJP: don't forget that writing test harnesses can be almost as much work or sometimes more as writing the actual code.

Delays during final testing are very demoralizing!

Gutless estimating

Urgency of boss forces programmers to agree to unrealistic schedules.

It is very hard to defend an estimate (good or bad); people use "hunches"

Surgical Team

The difference between a good programmer and bad programmer is at least:

• 10x in productivity
• 5x in program speed, space measurement

The 20k/yr programmer is more than 10x more productive than 10/yr programmer (1960's salaries...i hope) ;)

Data showed no correlation between experience and performance (but clearly there must be some).

"Small" team shouldn't exceed 10 programmers.

Managers want small sharp team, but you can't build a very large system this way.

OS360 took about 5000 man years to complete. A team of 200 programmers would take 25 years (assuming simple linear partitionable tasks) Took only 4 years with 1000 people (quoting from book these numbers but don't seem to add up).

Instead of hiring 200 programmers what about this: hire 10 superstars with say 7x productivity factor and 7x reduction in communication costs. 5000 hours / (10 x 7 x 7) = 10 years. Hmm...may not work even still.

Harlan Mills suggests surgical teams: team leader (chief programmer==surgeon) with supporting surgeons, nurses. Chief does the cutting and other support.

Might be tough to find right mix of people, desires, skills (i.e., who wants to do the testing?)

• Surgeon: chief programmer
• Co-pilot: able to do what surgeon does but is less experienced
• Administrator: handles money, people, space, machines, etc...)
• Editor: surgeon must do doc, but editor must clean it up
• Two secretaris
• Program clerk: maintaining technical records
• Toolsmith: serves surgeon's need for tools, utilities
• Tester: Devise system component tests, does debugging

What's the difference? Surgical team: project is surgeon's brainchild and they are in charge of conceptual integrity etc... In collaborative team, everyone is equal and things are "designed by committee". Causes chaos.

How to scale? Large system broken into subsystems assigned to surgical teams. Some coordination between surgical team leaders.

Aristocracy, Democracy, and System Design

Conceptual Integrity:

• better to have one good idea than many bad or uncoordinated nonstandard ideas
• very important aspect of a large system.
• user has to know just that main concept; whole project makes sense. For example, in UNIX everything is a stream (files, devices, tty, ...)
• preserved by architect that designs system top-down.

TJP: in my experience, having a single mind behind ANTLR has made all tools, concepts hold together well. Most projects are "touched" by many grad students as they drift through a department and work on the tool for a prof.

Ratio of functionality / conceptual complexity is important.

One or a few minds design, many implement (per surgical team).

Brooks argues for separating implementation by using clock (hands, etc...) design with the many implementations. Architecture is what happens, and implementation is how it happens.

Defending aristocracy he says: Even though implementators will have some good ideas, if they don't fit within the conceptual integrity, they are best left out.

Dangers of architects

• specs will be too rich and will not reflect practical cost considerations. A real danger.
• architects will get all the fun and shut out inventiveness of implementators. Brooks argues that implementation is fun too per previous arg.
• implementors will sit idle until specs become ready. Can wait to hire the implementors.

Second System Effect

First system tends to be small and clean. Knows he/she doesn't know everything and goes slowly.

As the system is built, new features occur to them. They record these ideas for the "next system."

With the confidence of having built the previous system, the programmer builds the second system with everything. Tendency is to overdesign. Cites the IBM 709 architecture that is an update to 704. So big that only 50% of features are used.

Another version of the effect is to refine pieces of code or features from old system that just aren't that useful anymore.

TJP: I tend to consider the next system to be functionally exactly the same but with a much better implementation. A few new features are ok. Actually ANTLR is less functional that old PCCTS!

To avoid can have concepts like feature x is worth m bytes and n ns of time.

Managers should hire chiefs that have at least two systems under their belt.

Communication in large project

How to communicate? In as many ways as possible.

• Informally; telephones and good idea of worker relationships.
• TJP: nowadays programmers use instant messaging a lot.
• Meetings; Regular project meetings, with teams giving briefings. Smoke out the little misunderstandings. Good way to check to see if you all have same ideas.
• Workbook; formal project workbook must be started.

Brook's Rules

1. A program is not a product nor system

2. Adding programmers to fix a delay only makes it take longer

3. Plan to throw one away, you will anyway. This book is ancient, but he says The only constancy is change itself and plan the system for change, which could come straight from the extreme programming books.

4. Second system affect: overdoing new feature list to overcome weakness in first system.

RXJS in Angular workshop

 I created this little workshop a while ago, feel free to check it out and learn something about Angular and RXJS!

https://github.com/nagyadam2092/rxjs-in-angular-workshop 

PS I updated this a long time ago, it uses Angular 7.

Angular runtime

Here are my notes on the topic of how the new Ivy renderer works, enjoy!

Angular Ivy tech talk

At the company where I work (keylight GmbH) I gave a tech talk about the new version of Angular, here are the slides, enjoy!

My current favourite trick in Node.js

At my current job I need to work a lot with Node.js backend services, and one thing is recurring - I need to simulate some delays in the code, mostly for testing purposes.

In the old days this was pretty problematic, we had the classic "callback hell", which made the code very unreadable, sometimes had to refactor (copy-paste mostly) huge parts of the codebase, just to simulate some delay.

Nowadays the air is more fresh regarding asynchronous programming in the JavaScript economy, because we have Promises supported natively, and in newer versions of Node.js you get async-await, which we learned about in one of my previous posts (don't get fooled, the post is about the package `co`, which was essentially the equivalent of today's async-await).

So how do we simulate delays today? It's super easy actually, you just need to know that async-await is basically operating on top of Promises. Without any further discussion, here is the code:

await new Promise(resolve => setTimeout(resolve, 5000));

Let's look at what this codesnippet does.

await is going to wait (no pun intended) for a Promise, and until it resolves, it's not going to continue the execution flow.

So we create the Promise, where we just wait using the setTimeout method 5000 milliseconds (5s), and then resolve the Promise! Simple as that!

I think this is a pretty handy codesnippet, that I wish I knew before!