I understand the difference between syntax and semantics -
Syntax: how the symbols are combined to form a valid expression or statement.
Semantics: the meaning of those symbols that form an expression or statement.
But what is the grammar? For example: sometimes I hear people say that some construct is "grammatically incorrect but syntactically it is correct". What does it mean?
Syntax just refers to how you express ideas, independent of the underlying ideas themselves.
A grammar is a set of (mostly syntactical) rules about how you form valid statements in a particular language, and the "type" of a statement, based on which rules were used to form that statement.
For example, C++ and Java have similar syntax in many respects, but completely separate grammars. Somebody who can read code in one can probably read code in the other with only minimal difficulty. But being able to read code reasonably well doesn't mean they can write it anywhere close to as well. For example, a C++ programmer might try to use a typedef, not realizing that Java has neither typedef nor anything even roughly equivalent (so in that respect, their syntax is completely different).
The difference is fuzzy and not worth worrying about too much.
People will sometimes include context-sensitive constraints under the umbrella of syntactic correctness. The most common example is a type system. Another is Java's "no statements after return" rule. This simplifies formal discussion: the syntax yields a language (a set of sentences/expressions/programs) which is the domain of the semantics; anything else is "not a program", and the semantics need not bother with it.
In contrast, "grammar" typically refers to a method of describing context-free languages (attribute grammars notwithstanding).
The reason it's not worth worrying about much is that type systems are as often considered the "static semantics" of a language as they are a "syntactic discipline for correctness". And sometimes a language doesn't quite have a proper context-free grammar; C, for example, must feed information from the parser back into the lexer.
Pragmatically, anyone who relies on a distinction between "syntactic" and "grammatical" had better say so and explain what they mean.
A grammar is a set of formulas that describe a syntax. There are a number of ways one can do this ranging from regular expressions (for extremely simple languages), to context free grammars, to far less common options.
Similarly, there are a number of methods you can use to describe semantics. Denotational, operational and axiomatic semantics being classes of methods for semantics.
A grammar is a set of rules to define a language. Rather, the grammar describes the syntax and semantics. A language might have two different grammars:
For example, a part of the grammar in C might look something like:
if statement -> if_keyword "(" expression ")" if_block
if_keyword -> "if"
logical_statement -> some other stuff here...
Meaning:
an if statement is made of an if keyword followed by a parenthesis followed by an expression followed by a parenthesis followed by an if block
an if keyword is ....
Take a look at this way of defining a grammar. If you're really curious about grammars, take a look at GNU Bison, which is basically a tool for describing the grammar of a language.
The "grammatically incorrect but syntactically correct" doesn't make too much sense. Maybe they're referring to a grammar that describes the semantics of a language. It would certainly make more sense to just say "not semantically correct" though.
When you are comparing programming languages to human languages (which the OP is implicitly doing), the best way to think about it is this:
In computer science explicitly, source code is tokenized. For example, there may be a single numerical value that represents a keyword, and a separate one for an open parenthesis. Computers are not restricted in what can be considered a token like human languages are.
The string of tokens then has to conform to certain patterns in order to generate the machine code that the CPU understands. For example, the grammar of a language dictates whether object.verb() is meaningful in the language, and what should happen.
When the parser cannot generate a valid token for any given part of the source code, the parser itself will generate a syntax error. If done well, it will also include enough information to find the offending code (for example a line number).
When you have valid syntax, your lexer will attempt to make sense of those tokens and generate the machine code. If there is an error in this stage, you will commonly get a generic compilation error (hopefully with a line number).
