Tag: floating-point

58 votes
12 answers
932 views
Why doesn't GCC optimize a*a*a*a*a*a to (a*a*a)*(a*a*a)?

I am doing some numerical optimization on a scientific application. One thing I noticed is that GCC will optim

avatar
Venice
0 1 0 9
asked 07 September, 2021
52 votes
3 answers
1.2K views
Why does Python's hash of infinity have the digits of π?

The hash of infinity in Python has digits matching pi: >>> inf = float('inf') >>>

avatar
Xavier
0 1 0 9
asked 07 September, 2021
62 votes
7 answers
942 views
Why is division in Ruby returning an integer instead of decimal value?

For example: 9 / 5 #=> 1 but I expected 1.8. How can I get the correct decimal (non-integer) result? Wh

avatar
Robt
0 1 0 9
asked 07 September, 2021
63 votes
21 answers
2.4K views
Why can't decimal numbers be represented exactly in binary?

There have been several questions posted to SO about floating-point representation. For example, the decimal n

avatar
Renaldo
0 1 0 9
asked 07 September, 2021
61 votes
30 answers
1.4K views
Is floating point math broken?

Consider the following code: 0.1 + 0.2 == 0.3 -> false 0.1 + 0.2 -> 0.30000000000000004

avatar
Dwain
0 1 0 9
asked 07 September, 2021
53 votes
6 answers
1.4K views
Why is 24.0000 not equal to 24.0000 in MATLAB?

I am writing a program where I need to delete duplicate points stored in a matrix. The problem is that when it

avatar
Arline
0 1 0 9
asked 07 September, 2021
55 votes
4 answers
1.3K views
Function for 'does matrix contain value X?'

Is there a built in MATLAB function to find out if a matrix contains a certain value? (ala PHP's in_array

avatar
Majorie
0 1 0 9
asked 07 September, 2021
50 votes
2 answers
742 views
In MATLAB, are variables REALLY double-precision by default?

This question arose out of something strange that I noticed after investigating this question further... I al

avatar
Dusty
0 1 0 9
asked 07 September, 2021
57 votes
29 answers
1.5K views
How do I parse a string to a float or int?

In Python, how can I parse a numeric string like "545.2222" to its corresponding float value, 545.2222? Or par

avatar
Carmelina
0 1 0 9
asked 07 September, 2021