Software Engineering Stack Exchange
Software Engineering Stack Exchange

Questions tagged [arithmetic]

30 questions
61
votes
5 answers

Will a computer attempt to divide by zero?

We all know 0/0 is Undefined and returns an error if I were to put it into a calculator, and if I were to create a program (in C at least) the OS would terminate it when I try to divide by zero. But what I've been wondering is if the computer even…
Ankush
  • 821
  • 1
  • 8
  • 12
15
votes
3 answers

Is there something special about the number 65535?

2¹⁶-1 & 2⁵ = 2⁵ (or? obviously ?) A developer asked me today what is bitwise 65535 & 32 i.e. 2¹⁶-1 & 2⁵ = ? I thought at first spontaneously 32 but it seemed to easy whereupon I thought for several minutes and then answered 32. 32 seems to have been…
Niklas Rosencrantz
  • 8,008
  • 17
  • 56
  • 95
10
votes
2 answers

Name of the Countdown Numbers round problem - and algorithmic solutions?

For the non-Brits in the audience, there's a segment of a daytime game-show where contestants have a set of 6 numbers and a randomly generated target number. They have to reach the target number using any (but not necessarily all) of the 6 numbers…
Dai
  • 680
  • 4
  • 15
9
votes
3 answers

Is order of arguments in an arithmetic expression important to achieve as most exact result as possible (speed is not necessary)?

Actually, in this question I do not ask about particular language or architecture, however I understand there might be some differences. In physics / engineering it is usually better to handle greater numbers than smaller ones, because of better…
Voitcus
  • 201
  • 2
  • 5
9
votes
3 answers

What rationale is used when programming language designers decide what sign the result of modulo operation takes?

Going through Modulo operation (the avenue I entered while exploring the difference between rem and mod) I came across: In mathematics the result of the modulo operation is the remainder of the Euclidean division. However, other conventions are…
Bleeding Fingers
  • 343
  • 1
  • 12
8
votes
7 answers

What is the size of the number 65535 in bytes?

As I got to know there are 256 possible combinations to get for 1 byte. If I understand it correctly, it should mean that you can display any number out of numbers 0-255 and this very number would use only 1 byte of your storage on your computer.…
treesarecool
  • 151
  • 1
  • 4
6
votes
2 answers

Performance and other issues with using floating point types in C++

Being interested in C++ performance programming there is one aspect I really have no clue about- and that is the implications of using floating point calculations vs doubles vs normal integer mathematics? What performance issues with these types of…
user997112
  • 1,469
  • 2
  • 19
  • 24
5
votes
2 answers

Array Multiplication and Division

I came across a question that (eventually) landed me wondering about array arithmetic. I'm thinking specifically in Ruby, but I think the concepts are language independent. So, addition and subtraction are defined, in Ruby, as such: [1,6,8,3,6] +…
Narfanator
  • 222
  • 2
  • 6
4
votes
1 answer

Semantics of simulating a 64bit integer with two 32bit integers (hi, lo)

I am working in a system which only supports 32bit integers, in order to have a 64bit (unsigned) integer I decided to simply use two 32bit integers with one being the upper 32 bits (hi), and the other being the lower 32 bits (lo). I also needed to…
Prime
  • 141
  • 1
  • 4
4
votes
4 answers

Why are arbitrary-precision decimals used over rational numbers in practice?

The basic arithmetic a computer chip can do only works on numbers(integers or floating point) of a fixed size. There are many algorithms that could be extended to work on numbers of arbitrary size (and some require them). Therefore, a way to store…
Qqwy
  • 4,709
  • 4
  • 31
  • 45
4
votes
2 answers

What do you get when you cross a multi-precision integer with a floating-point number?

I'm working on adding multi-precision integers to the suite of numeric types in my APL interpreter, but I'm not sure what to do about the odd type-combinations that now arise. I now have the following types: IMM "atomic" small integer FIX full-width…
luser droog
  • 419
  • 4
  • 12
4
votes
1 answer

how to enumerate/generate all possible binary trees from N leaves and N-1 nodes?

I am trying to implement the 24 Game in ansi C. This game goes as follows: For a list of four given numbers, try to find a solution involving these four numbers, that, using addition(+), subtraction(-), multiplication(*) and division(/) ends up at…
Qqwy
  • 4,709
  • 4
  • 31
  • 45
4
votes
2 answers

Arithmetic Coding vs. Machine Numeric Precision

When subdividing intervals for arithmetic coding, the worst case scenario is that the final interval will have size 2^(1-n), where n is the number of unique symbols you are encoding. This will reach the limits of machine precision, when using…
Xophmeister
  • 849
  • 1
  • 7
  • 11
3
votes
1 answer

Difficulties with Two's complement in Assembly

Everything discussed here will refer to Intel 8085 (8-bit architecture). When using two's complement number "conversion" : 1.) If we take SBI 0F (subtraction) for example; immediate value 0F(hex) or 15 is subtracted from A register, which is loaded…
Lucenzo97
  • 293
  • 4
  • 11
3
votes
2 answers

Arithmetic Coding Questions

I have been reading up on arithmetic coding and, while I understand how it works, all the guides and instructions I've read start with something like: Set up your intervals based upon the frequency of symbols in your data; i.e., more likely symbols…
Xophmeister
  • 849
  • 1
  • 7
  • 11
1
2