Software Engineering Stack Exchange
Software Engineering Stack Exchange

Questions tagged [indexing]

69 questions
62
votes
10 answers

Is it premature optimization to add database indices?

A colleague of mine today suggested that we go through all of the queries in our application and to add indices accordingly. I feel this is premature optimisation because our application is not even released yet. I suggested to monitor for slow…
Marco de Jongh
  • 897
  • 6
  • 13
48
votes
2 answers

What is the difference between btree and rtree indexing?

I've noticed on MySQLWorkbench that you can choose how to store your indexes before forward engineering your design. The storage types are: BTREE RTREE HASH Researching this, I found some information that was pretty much over my head, so I'm…
user25791
36
votes
7 answers

How to quickly search through a very large list of strings / records on a database

I have the following problem: I have a database containing more than 2 million records. Each record has a string field X and I want to display a list of records for which field X contains a certain string. Each record is about 500 bytes in size. To…
Giorgio
  • 19,486
  • 16
  • 84
  • 135
25
votes
8 answers

Should my sequential collection start at index 0 or index 1?

I am creating an object model for a device that has multiple channels. The nouns used between the client and I are Channel and ChannelSet. ("Set" isn't semantically accurate, because it's ordered and a proper set isn't. But that's a problem for a…
kdbanman
  • 1,447
  • 13
  • 19
14
votes
6 answers

Do You Still Need Indexing After Database Normalization

After you have done a good normalization, do you still need to index the table? How is this going to affect the performance? Will it even affect the performance in some way after a good normalization? Which columns are usually indexed if you…
Franz Noel
  • 303
  • 4
  • 12
12
votes
5 answers

How come the computer doesn't have to read the entire table when the column is indexed?

Let's say a table with two columns has 100 quadrillion records. And I want to find a record that has column #2 equal something. If column #2 is indexed it returns the result immediately, but if it's not the computer has to read the entire table so…
user1806244
  • 167
  • 7
11
votes
5 answers

Quadtree with duplicates

I'm implementing a quadtree. For those who don't know this data structure, I am including the following small description: A Quadtree is a data structure and is in the Euclidean plane what an Octree is in a 3-dimensional space. A common use of…
Pierre Arlaud
  • 1,329
  • 1
  • 13
  • 21
10
votes
2 answers

Is indexing foreign keys a good practice?

Looking at DB tables created by a different developer I have noticed that whenever a table had a forein_key_id field/column, it was always an INDEX/KEY. I am not sure if it was manually created, or automatically by some 3rd party software. …
Dennis
  • 8,157
  • 5
  • 36
  • 68
10
votes
1 answer

lightweight document indexing to handle less than 250k potential records

Recently I've found myself chafing at the limitations of document indexing engines. I was developing a small website that needed some fairly robust searching capabilities but due to their hardware constraints I couldn't deploy a Lucene-ish solution…
Jarrod Nettles
  • 6,125
  • 2
  • 41
  • 45
9
votes
4 answers

What is the origin of counting from zero in programming languages?

This is a question which I have wondered (and been asked) about for a long time. In (most? all?) programming languages, an index begins at zero for an array, string, etc. I recognize it became convention over time, adopted in many languages, but can…
erinbrown
8
votes
1 answer

How do I efficiently store all OpenStreetMap data in an indexed way?

I have a PBF file that contains the following information about a country: Nodes, each with their own longitude, latitude and properties; used to store points in a 2D space. Ways, each with their properties, they are connected through nodes; used…
Tamara Wijsman
  • 8,259
  • 14
  • 58
  • 94
8
votes
1 answer

Database design for tagging system

I'm creating a system where items will have one or more tags. Now there will be an item table that will store each item, and a tags table, that will store all tags used in the system. My question is: What is the best design I can have that will…
Ron
  • 183
  • 1
  • 1
  • 6
7
votes
4 answers

Can someone explain the technicalities of MapReduce in layman's terms?

When people talk about MapReduce you think about Google and Hadoop. But what is MapReduce itself? How does it work? I came across this blog post that tries to explain just MapReduce without Hadoop, but I still have some questions. Does MapReduce…
Eddie Bravo
  • 81
  • 3
5
votes
3 answers

Is "Array[1]" the first element or second element in the array?

Following the reading of the question Why are zero-based arrays the norm?, I wonder about the terms to use for referring to specific array elements, in the perspective of linguistic reading of programming. Should Array[1] be called the first element…
profaisal
  • 97
  • 1
  • 5
5
votes
4 answers

Approach for parsing and indexing very large files

I have been tasked with developing a web based (i.e runs in browser) viewer for a proprietary log file. I have no control over the format of the logs, I just consume them. The log file contains binary data appended by a text message on each line, so…
Matt
  • 255
  • 2
  • 7
1
2 3 4 5