Why doesn't reduce() throw a Runtime exception on an empty stream instead of returning the identity?

a) Stream.reduce()
T reduce(T identity, BinaryOperator<T> accumulator)
returns identity on an empty stream.
So for instance, this is an incorrect result returned.

       Stream<Integer> stream = Stream.of();        System.out.println(stream.reduce(1, Math::multiplyExact));

Output:

1

I understand that

b)
// Optional<T> reduce(BinaryOperator<T> accumulator)        stream = Stream.of();        System.out.println(stream.reduce( Math::multiplyExact));
Output:

Optional.empty

will return an empty Optional - which is the correct result.
However, if a) could throw a Runtime exception, then we would not have the risk of a silent error.
An insidious bug.
In a more elaborate reduce operation, it might not be so easy to spot that there was either an empty stream, or worse,
that the stream had elements but the identity was not the correct result.

The docs say

This means that for all t, accumulator.apply(identity, t) is equal to t.

I am wondering if there are other circumstances where the identity returned in the reduce in a) is incorrect?

 

I don't see why returning the identity value of the operator is incorrect. Maybe you could explain this?

Paul Clapham wrote:I don't see why returning the identity value of the operator is incorrect. Maybe you could explain this?

nothing times nothing is not equal to 1

unless I am missing something.

Anil Philip wrote:

Paul Clapham wrote:I don't see why returning the identity value of the operator is incorrect. Maybe you could explain this?

nothing times nothing is not equal to 1

unless I am missing something.

First, you aren't starting with nothing - you are starting with 1.  As your code specifies when you write stream.reduce(1, Math::multiplyExact)).  

Second, you aren't multiplying by nothing.  You are multiplying by a value in the stream, for each element in the stream.  If there are no values in the stream, then no multiplication occurs at all.  There's nothing to multiply.  So, you're just left with what you started with.  Which was 1.

Here's some code to look at that may help you feel better about it:
   for (int i = 10; i >= 0; i--) {      var stream = IntStream.generate(() -> 2).limit(i);      var result = stream.reduce(1, Math::multiplyExact);      System.out.println("for " + i + " elements, result is: " + result);    }
Output:
for 10 elements, result is: 1024 for 9 elements, result is: 512 for 8 elements, result is: 256 for 7 elements, result is: 128 for 6 elements, result is: 64 for 5 elements, result is: 32 for 4 elements, result is: 16 for 3 elements, result is: 8 for 2 elements, result is: 4 for 1 elements, result is: 2 for 0 elements, result is: 1

The identity of an operator * is the value  e such that e * x = x for every value x in the domain of that operator. That's how it's defined.

So 1 is the identity of multiplication and 0 is the identity of addition. (This means that the sum of an empty set of integers is 0 and the product of an empty set of integers is 1.)

Not all operators have an identity -- you'll find that subtraction doesn't have one, for example. That's why the documentation mentions that operators must be associative if you're providing their identity. Also, providing an incorrect identity will produce weird results.

For an exercise, find out what the identity of Math.max is.

https://en.wikipedia.org/wiki/Identity_element

In mathematics, an identity element or neutral element of a binary operation is an element that leaves unchanged every element when the operation is applied.

a *1 = a
1 * 1 = 1
a + 0 = a

Paul Clapham wrote: (This means that the sum of an empty set of integers is 0 and the product of an empty set of integers is 1.)

Do you have a source for this?

Mike Simmons wrote: If there are no values in the stream, then no multiplication occurs at all.  There's nothing to multiply.  

Then there is no result. Please also see my reply to Paul above

Just so you know, Paul C got his PhD in mathematics from Cal Tech. It might help to spend less time reflexively disagreeing, and more time absorbing. Also, your own source, wikipedia, is telling you in the article that you quoted, that the identity for the addition operation is 0, and that for multiplication is 1.  Maybe Paul is onto something here...

Anil Philip wrote:Then there is no result.

Well, you're using a method that documents pretty clearly how it works.  Have you looked at the API for Stream.reduce() with two arguments?  It says the result should be equivalent to:
    T result = identity;     for (T element : this stream)         result = accumulator.apply(result, element)     return result;
In this code, if you have a stream with no elements, what is the result?

The whole idea of Streams was to provide code that would run normally to completion. Why would you want to throw an exception?

Campbell Ritchie wrote:The whole idea of Streams was to provide code that would run normally to completion. Why would you want to throw an exception?

At least it's better than providing an incorrect result.
If I see identity as the result, it is ambiguous in the current way.
There could have been zero to many elements in the stream, resulting in identity.
If an exception is thrown, at least we know that the operation never happened.

Mike Simmons wrote:Just so you know, Paul C got his PhD in mathematics from Cal Tech.

That is awesome... to have such great pedigree. Perhaps he can give us a good explanation?

Mike Simmons wrote:
It might help to spend less time reflexively disagreeing, and more time absorbing.

No, I am asking what I believe to be a valid question, to try and understand.

Mike Simmons wrote:
Also, your own source, wikipedia, is telling you in the article that you quoted, that the identity for the addition operation is 0, and that for multiplication is 1.  Maybe Paul is onto something here...

But I don't see where it says that when you have no elements to perform the operation on, Identity is still the result.

Mike Simmons wrote:
Well, you're using a method that documents pretty clearly how it works.  Have you looked at the API for Stream.reduce() with two arguments?  It says the result should be equivalent to:
    T result = identity;     for (T element : this stream)         result = accumulator.apply(result, element)     return result;
In this code, if you have a stream with no elements, what is the result?

That is only telling me that Java is using the term 'identity' in a different way that Math uses it.

Anil Philip wrote:Perhaps he can give us a good explanation?

He did.  Whether he chooses to give another is up to him.

Anil Philip wrote:That is only telling me that Java is using the term 'identity' in a different way that Math uses it.

No, I don't think the problem is with identity at all - rather, it's with reduce.  The definition of identity doesn't say anything about what it's supposed to do with an arbitrary-length list or stream of elements - that's where the reduce operation comes into play.  We both seem to be happy enough with the Wikipedia article on identity, but we haven't agreed on what reduce should do.  I tried using Java's definition, which you had linked to previously, but you don't seem to like that now.  

In Wikipedia we have

Reduce_(higher-order_function)

which redirects to

Fold_(higher-order_function)

which reflects that outside Java, much of the Functional Programming community uses "fold" rather than "reduce".  The article is then somewhat dense with FP-specific terminology, so I'm not sure if you want to use that.  If not, what definition of reduce would you like to use?  What do you think it's supposed to mean?

Anil Philip wrote:That is awesome... to have such great pedigree. Perhaps he can give us a good explanation?

I already provided a good explanation.

Anil Philip wrote:But I don't see where it says that when you have no elements to perform the operation on, Identity is still the result.

Well, it doesn't specifically say that. It doesn't have to. When you take the product of a set of integers, you get an integer as the answer. And since the empty set is a set, the product of the empty set of integers has an integer value too. Mathematicians hate it when you make up special cases when there is no reason to do so, therefore you should treat the empty set the same way as a non-empty set if at all possible.

In Math you could write this theorem: If A and B are sets of integers then product(A) * product(B) = product(A followed by B). This is true whether or not A or B are empty so having to take into account something different when one or both sets is empty is just ugly because it's unnecessary. And mathematicians hate that sort of ugliness.

(Note: I wrote "followed by" instead of "union" so I avoided what would happen if an integer was in both A and B. If A is 1,4,6 and B is 2,4,12 then we're talking about 1,4,6,2,4,12.)

Anil Philip wrote:That is only telling me that Java is using the term 'identity' in a different way that Math uses it.

No, you have that backwards. You came here complaining that Java should throw an exception when asked to provide the product of an empty set of integers. But Java doesn't do that. It does what Math says it should do.

Paul Clapham wrote:
I already provided a good explanation.

Anil Philip wrote:But I don't see where it says that when you have no elements to perform the operation on, Identity is still the result.

Well, it doesn't specifically say that. It doesn't have to. When you take the product of a set of integers, you get an integer as the answer. And since the empty set is a set, the product of the empty set of integers has an integer value too. ...
No, you have that backwards. You came here complaining that Java should throw an exception when asked to provide the product of an empty set of integers. But Java doesn't do that. It does what Math says it should do.

no point in continuing this discussion since you cannot provide me with any accepted source that backs up what you are saying.
(because I never heard this before in any Math class),
but instead say that I am "complaining" and "reflexively disagreeing".

Anil Philip wrote:no point in continuing this discussion since you cannot provide me with any accepted source that backs up what you are saying.
(because I never heard this before in any Math class),

Where "accepted" source means one that you personally accept? I do remember that my Math 1 students often found this view of empty sets to be disconcerting, but it's mainstream Math.

but instead say that I am "complaining" and "reflexively disagreeing".

You said "So for instance, this is an incorrect result returned" and called it "an insidious bug". Neither statement is true. This looks like complaints to me but maybe you're a person who doesn't consider seriously negative statements to be "complaints".

Anil Philip wrote:. . . At least it's better than providing an incorrect result. . . .

What makes you think 0 is a wrong result for a sum or 1 for a product?

There could have been zero to many elements in the stream, resulting in identity. . . .

Like this, you mean?var sum     = IntStream.of(0, 0, 0, 0, 0, 0)                       .reduce(0, Integer::add); var product = IntStream.of(1, 1, 1, 1, 1, 1)                       .reduce(1, (i, j) -> i * j);What you are doing is reducing the Stream to a single value, which you can see clearly in IntStream#reduce(). When you reduce a Stream, you are losing information, including how many elements there were. That is normal behaviour for reducing; it does not care about any other information. The result would be the same for my IntStreams if you add more of the same elements. The limit to which the result tends as the Stream lengthens (with more of the same) is the same as the identity shown as the first argument in lines 2 and 4. If you shorten the Stream towards length 0, the limit towards which the result tends is also the identity (sometimes called the unit).
∴ The identity is the correct result from reducing a 0‑length Stream. The IntStream link I gave you says you must supply the appropriate identity.
It is like counting how much money you have in your pocket; you get a total and you forget whether it was in folding money or dollars or cents. You might be unlucky and not find any money at all. That is the identity or unit for addition: 0. When I go shopping, I pay the total price and don't worry about how many items I am paying for, assuming I am sure there has been no mistake in the counting.
And remember, after all the confusion I caused on your other thread, I showed you an example where a 0‑length Stream is a best practice option.
 * * * * * * * * * *
Note I have shown the behaviour defined by the two‑argument version of the method. Its name is overloaded and you might prefer to use the one‑argument version, which has a different return type and allows you to tell whether you started with a 0‑length Stream. The returned result isn't null, nor does the reduce() method throw an exception.

Man, I would be so upset if multiplying zero integers would return anything other than 1. Throwing an exception is what I would call "an insidious bug".

Anyway, seeing as you hold Wikipedia in high regard as an authoritative source, here's a link for you: https://en.m.wikipedia.org/wiki/Empty_product

Just a word of advice. Your learning journey is going to be very tough if you insist on a source for everything we tell you. Stop and think about what you're doing:

You've joined a forum so that you can have your questions answered by experts who collectively have centuries of knowledge and experience; some, like Paul, directly at the very edge of human knowledge of a specific subject.

Now, instead of getting on with your life and trying to digest what we, a whole forum of experts, are telling you, you insist that we must be wrong because you, an intermediate, hasn't heard about it in school.

I'm not saying that scepticism isn't good, and I'm not saying that we are free from backing up our statements with good citations every now and then; but if EVERYONE else is agreeing with each other and is telling you that something works a certain way and that it is for a good reason, maybe just take the hint.

Stephan van Hulst wrote:. . . multiplying zero integers would return anything other than 1. . . .

If you multiply x by itself n times, you will get xⁿ, so when you multiply it zero times, you should get x⁰, and for all finite values of x, that comes to 1, except in some cases if x = 0.

Stephan van Hulst wrote:here's a link for you: https://en.m.wikipedia.org/wiki/Empty_product

I noticed in that page a reference to "vacuous truth", which I remember from my Math 1 TA days because my students specifically asked me about the formulation given there: "empty set of objects can have any property". That's unfortunately misleading; for example it isn't true that an empty set has the property of containing seven elements. It's better to state it in terms of the elements of an empty set; statements about elements of any empty set are true. For example, "all of the crocodiles in my bathroom are blue" is a true statement because there are no crocodiles in my bathroom.

Paul Clapham wrote:. . . there are no crocodiles in my bathroom.

Are you sure about that?
Vacuously true things are true, but you can end up with a program like “Magic,” which can legitimately promise anything because it never has to deliver.
“Magic” is defined as false ⟶ skip where the ⟶ is the guard operator. Its weakest precondition to establish any postcondition Q turns out to be vacuously true because the weakest precondition turns out to be false ⇒ skip[Q] and we all know that false ⇒ anything is true. Vacuously so.

Campbell Ritchie wrote:

Paul Clapham wrote:. . . there are no crocodiles in my bathroom.

Are you sure about that?

I think Paul C is on pretty safe ground there.  If it was Tim H, someone might need to go check to be sure.  Though the answer might change with global warming...

Campbell Ritchie wrote:

Paul Clapham wrote:. . . there are no crocodiles in my bathroom.

Are you sure about that?

I suppose I should get used to having my assertions questioned here. No, unfortunately I can't point you to a reference on the web confirming that.

Paul Clapham wrote:For example, "all of the crocodiles in my bathroom are blue" is a true statement because there are no crocodiles in my bathroom.

Genuine question: By the above logic, I could also say, "none of the crocodiles in my bathroom are blue", right?

Let's convert that natural language statement to first-order logic:

¬∃𝑥∊𝓧 𝑃(𝑥)

Here, 𝓧 is the set of crocodiles in my bathroom, and 𝑃(𝑥) is the predicate that crocodile 𝑥 is blue.

The first-order logic statement above can be read as: "There does not exist at least one crocodile in my bathroom for which its color is blue".

The quantified function ∃𝑥∊𝓧 𝑃(𝑥) trivially returns false if 𝓧 is the empty set ∅. So if we negate it with the ¬ operator, it returns true for the empty set.

As long as there are no crocodiles in our bathroom, both of the following statements hold true simultaneously:

"all of the crocodiles in my bathroom are blue"

"none of the crocodiles in my bathroom are blue"

Paul Clapham wrote:And since the empty set is a set, the product of the empty set of integers has an integer value too.

Stephan van Hulst wrote:Man, I would be so upset if multiplying zero integers would return anything other than 1. Throwing an exception is what I would call "an insidious bug".

I was watching a video on Russel's paradox on youtube (https://www.youtube.com/watch?v=ymGt7I4Yn3k&t=698s) and at around 11.50, while explaining empty sets, he says that there is exactly one empty set and that two empty sets are really the same set.

If he is right, an empty set is as much a set of Integers as it is a set of monkeys. So, it is not entirely implausible to think that multiplying the elements of an empty set of Integers should be undefined, in which case an RTE is more appropriate.

Not really questioning the logic behind the reduce(U identity,  BiFunction<U,? super T,U> accumulator, BinaryOperator< U> combiner) because returning identity for an empty set is also valid from one perspective, but since they already have so many methods, they could have easily had one that didn't take an identity parameter and threw an RTE for empty stream as well i.e., reduce( BiFunction<U,? super T,U> accumulator, BinaryOperator< U> combiner).

(Fixed formatting)

Paul Anilprem wrote:. . . there is exactly one empty set and that two empty sets are really the same set. . . . an empty set is as much a set of Integers as it is a set of monkeys. . . .

That is the correct mathematical abstraction, but programmers have always bent the rules. For example, the double datatype supports the non‑existent number -0.0. Generics might allow a set containing no monkeys to be equal to a set containing no integers, but you can't interconvert the set by adding an integer to it. This reduce() method requires the user to supply an identity, 1 for multiplication or 0 for addition, but is there an identity appropriate for subtraction?
The reason for not throwing an exception is that the designers didn't want Streams to fail to complete their runs.

Campbell Ritchie wrote:
The reason for not throwing an exception is that the designers didn't want Streams to fail to complete their runs.

Right. I should rephrase it as:
"So, it is not entirely implausible to think that multiplying the elements of an empty set of Integers should be undefined, in which case an RTE is more appropriate returning an empty Optional is more appropriate. "

What I am trying to get at is that returning Identity makes the method more rigid than what is warranted. It doesn't allow the the programmer to have an alternate path if there are no elements in the stream.  Given that there already are a buttload of methods, what's one more? Unless it is doesn't make sense logically.

Paul Anilprem wrote:. . . returning an empty Optional is more appropriate.

Isn't that what happens if you use the method without the identity parameter?

. . . there already are a buttload of methods, what's one more?

Agree there

Unless it is doesn't make sense logically.

That never stopped them making StringBuilder implement Comparable even though it doesn't override equals().

Campbell Ritchie wrote:Isn't that what happens if you use the method without the identity parameter?

No, the one that doesn't take identity parameter takes one parameter of type BinaryOperator. A BinaryOperator restricts you to returning the same type as the type of the stream.
In fact, now I wonder why did they not design this method to accept a BiFunction like the other reduce method.

Paul Anilprem wrote:

Campbell Ritchie wrote:Isn't that what happens if you use the method without the identity parameter?

No, the one that doesn't take identity parameter takes one parameter of type BinaryOperator. A BinaryOperator restricts you to returning the same type as the type of the stream.
In fact, now I wonder why did they not design this method to accept a BiFunction like the other reduce method.

Because that would require a combiner, of course!

I meant something slightly different, that the one‑argument version of that method returns OptionalInt.
If we are thinking of the IntStream method, the logic (I think) is that in an IntStream you are sticking to ints, and if you want a different datatype, you use a mapToXXX() method.
Another reason, maybe, for not throwing exceptions from an empty Stream is that exceptions are supposed to respond to something abnormal, and an empty Stream is regarded as a normal occurrence.

Campbell Ritchie wrote:
Another reason, maybe, for not throwing exceptions from an empty Stream is that exceptions are supposed to respond to something abnormal, and an empty Stream is regarded as a normal occurrence.

Yes, I agree with that. It seems reasonable.

What I am not sure about now is the reason behind having the following three methods in the Stream interface (i.e. the regular Stream interface and not the primitive stream interfaces such as IntStream) :

Optional<T> reduce(BinaryOperator<T> accumulator)

T reduce(T identity, BinaryOperator<T> accumulator)

V reduce(V identity, BiFunction< V,? super T,V> accumulator, BinaryOperator<V> combiner)

but not having:

Optional<V> reduce(BiFunction<V,? super T, V> accumulator, BinaryOperator combiner)

Paul Anilprem wrote:but not having:

Optional<V> reduce(BiFunction<V,? super T, V> accumulator, BinaryOperator combiner)

With this method, how would you ever get a V instance at all?  You have a stream of T, and you have a BiFunction to take a V and a T, and return a V.  Which means you need at least one V to use that function.  That's why you need an identity V instance, to start the reduction off.  

And for parallel processing, each worker thread will use that same identity V instance to start off processing its own subset of elements, before combining later with the combiner.  And if any of those worker threads finds there are no elements for them to process (maybe it turns out that there's a filter() method that eliminates all the elements that might have been expected) you want the worker to not freak out and throw an exception, just return the identity value unmolested, so it can still be combined with the other results safely.

Campbell Ritchie wrote:This reduce() method requires the user to supply an identity, 1 for multiplication or 0 for addition, but is there an identity appropriate for subtraction?

No.  Subtraction is not a suitable reduction operation in the first place, because it is not associative - i.e.

(a-b)-c

is not the same as

a-(b-c)

Associativity is one of the properties required of the accumulator, in order to get a meaningful result.

Mike Simmons wrote:. . . No. . . . .

I thought not; I would have been sure if I had bothered to read the documentation.

Also covered by Dr. Clapham a few months ago, above:

Paul Clapham wrote:Not all operators have an identity -- you'll find that subtraction doesn't have one, for example. That's why the documentation mentions that operators must be associative if you're providing their identity. Also, providing an incorrect identity will produce weird results.

Although I don't really see a compelling use case for it, in theory I can buy the idea that some people might want to make a stream that fails if there are no elements in it.  As it happens, I was looking at Java 22's new Stream Gatherers feature, which purports to be a mechanism for adding arbitrary new functionality to streams as an intermediate method.  Much like Collectors can add arbitrary new features to Stream, except that Stream.collect(Collector) is a terminal operation, while Stream.gather(Gatherer) is an intermediate operation.  So, it's a good candidate for the use case here, which is to add fail-on-empty-stream functionality to a Stream without interfering with whatever other operations you wanted the Stream for.  Here's what I came up with:
import java.util.stream.Gatherer; import java.util.stream.Stream; public class PickyStreams {  public static <T> Gatherer<T,?,T> requireNotEmpty() { return requireMinimumCount(1); }  public static <T> Gatherer<T,?,T> requireMinimumCount(long min) {    class Accumulator { long count; }    return Gatherer.of(      Accumulator::new,      (a, t, d) -> { a.count++; d.push(t); return true; },      (a1, a2) -> { a1.count += a2.count; return a1; },      (a, d) -> {        if (a.count < min)          throw new RuntimeException("Stream required " + min + " elements, found only " + 0);      }    );  }  public static void main(String... args) {    System.out.println(      Stream.of(2,5,9)        .gather(requireNotEmpty())        .reduce(1, Math::multiplyExact) // result: 90    );    System.out.println(      Stream.<Integer>of()        .gather(requireNotEmpty())        .reduce(1, Math::multiplyExact) // result: throws NoSuchElementException    );  } }
This requires Java 22, and further requires enabline preview features:
java --enable-preview --source 22 PickyStreams.java
But yes, it does work.  So, you may have to wait until September 2025 or so for a non-preview LTS implementation, but eventually, Java will allow this sort of arbitrary restriction to be inserted into a Stream.  Yay?  

Well, I was mostly interested in getting a handle on Gatherers, and this is a fairly simple case for that.