[Release b]
<t4m_bug_arraybound>
Thu May 13 10:44:23 WEST 2004

In the current release of tendra4minix (a), after fixing
<t4m_bug_copy_func_args>, the compilation of the file arraybound.C produces
this error

"arraybound.C", line 14: Error:
  [ISO 14.7.1]: In instantiation of template 'Adios < double >::Adios' (at line 19).
  [ISO 5.19]: Illegal integer constant expression.
  [ISO 8.3.4]: Array bound should be an integer constant expression.

This error is reported in the function expand_type1 [token.c] (the former
expand_type, that we turned into expand_type1 because of <t4m_bug_tok_recur>),
in the case type_array_tag (it is ERR_decl_array_dim_const). The problem
disappears when we change 'rows' for 'n' in the call to 'operator new' (see
arraybound_e.C). It's not because the expression occurs inside a constructor.
It also happens inside an ordinary function. It does not happen if we change
the type of the array elements from Hola<T> to T (see arraybound_t.C).

We see that the cause is that an error ERR_expr_const_bad is generated just
before, and the only place where this error is generated is the function
make_nat_exp [../parse/constant.c], in the cases type_enumerate_tag,     
type_bitfield_tag and type_integer_tag (I guess this is the one involved).

Ascending call trace::

expand_type [token.c], type_array_tag
  expand_nat [token.c], nat_calc_tag
    make_nat_exp [../parse/constant.c], type_integer_tag
      is_const_exp [check.c], c = 0, returns 0

Comparing the program that is compiled correctly (arraybound_e.C) with the one
that gives the error (arraybound.C), we see that the problem is that the
function make_nat_exp [../parse/constant.c] should not be invoked.

The function expand_type [token.c] is being called from copy_exp [copy.c] (as
usual) to instantiate the constructor Adios<double>::Adios. The type t that is
being expanded is the array type that we have used as argument for the operator
new. First, the type(_array_sub) s1 of the array elements is expanded and saved
in s2. Then, the (type_array_)size n1 of the array is expanded by calling
expand_nat. In the program that is compiled correctly (arraybound_e.C), this
field is an expression made of two nodes,

  exp_contents_tag
    exp_identifier_tag, exp_identifier_etc_id(id) = "n"

(i.e., "the contents of the identifier n"). In the program that is not
compiled (arraybound.C), this field is a complex expression that gives the
contents of a certain memory location with respect to the contents of the
'this' pointer,

  exp_contents_tag
    exp_indir_tag
      exp_add_ptr_tag
        exp_contents_tag
          exp_identifier_tag, exp_identifier_etc_id(id) = "<this>"

For both programs, the function expand_nat [token.c] reaches the case

	case nat_calc_tag : {
	    /* Calculated values */
	    EXP e2 ;
	    EXP e1 = DEREF_exp ( nat_calc_value ( n ) ) ;
	    ulong tok = DEREF_ulong ( nat_calc_tok ( n ) ) ;
	    if ( rec ) {
		e2 = eval_exp ( e1, ch ) ;
	    } else {
		e2 = expand_exp ( e1, 0, 0 ) ;
	    }
	    e2 = convert_reference ( e2, REF_NORMAL ) ;
	    e2 = convert_lvalue ( e2 ) ;
	    if ( !EQ_exp ( e1, e2 ) && !eq_exp_exact ( e1, e2 ) ) {
		n = make_nat_exp ( e2, err ) ;
		if ( IS_nat_calc ( n ) ) {
		    COPY_ulong ( nat_calc_tok ( n ), tok ) ;
		}
	    }
	    break ;
	}

and things go on in the same way for both programs: the function eval_exp
[copy.c] is called, and it reaches its 'default' case, where it calls copy_exp
[copy.c]. The calls to convert_reference [convert.c] and convert_lvalue
[convert.c] have no effect, returning their arguments. The call
'EQ_exp ( e1, e2 )' [../obj_c/c_class.h] fails for both programs, since it is
a pointer comparison, and the expressions e1, e2 have been duplicated by
copy_exp. The first difference between the compilation of the programs
arraybound.C and arraybound_e.C is that the call to the function eq_exp_exact
[check.c] succeeds for arraybound_e.C (the one that works), as it should be,
while it fails for arraybound.C.

I have traced the calls below eq_exp_exact [check.c], and it calls to eq_exp
[check.c], which in turn calls eq_exp_aux [check.c]. What is failing in
arraybound.C is the comparison of the respective 'this' pointers in the leaf
nodes of the expressions. This is because the comparison of the identifiers is
done by the macro EQ_id [../obj_c/c_class.h], which compares just pointer
values, and it seems that the call to the function rescan_member [copy.c]
(that is made in the case exp_identifier_tag inside copy_exp [copy.c]) has
built another instance of the 'this' identifier instead of reusing it.

I think that both rescan_member [copy.c] and eq_exp_aux [check.c] are working
properly. On entering rescan_member, the type of the 'this' identifier is
'Adios &', while on exiting it is 'Adios<double> &', and this is ok since the
task of copy_exp and expand_type is instantiating templates.

What is suspicious is the invocation of make_nat_exp. This function accepts
only constant expressions, and it is obviously suited only for the declaration
of static/automatic arrays, not dynamic arrays. Now it is evident that the
example arraybound_e.C works well just by chance, since the array size 'n' does
not depend on the template parameters; if it depended on the template
parameters in any way (as it is the case of the data member 'rows', whose
access involves a dependent 'this' pointer), then the function eq_exp_aux
[check.c] will fail because of the instantiation of the template parameters,
and this is what is happening with arraybound.C. Another example of this
situation is the program arraybound_f.C, which gives the same error, and this
time the size of the array is not a data member of any class, just an ordinary
function parameter.

To understand the problem, consider this distinction: a 'new' expression, that
is, an expression based on the operator 'new', is not the same as a call to the
(overloaded) function 'operator new'. The functions copy_exp [copy.c] and
expand_type [token.c] are not meant to process calls to 'operator new', they
can only deal with 'new' expressions. Let's begin by observing the following
descending call trace that starts directly from the parser,

new-expression [cpp/syntax/syntax.sid]
  new-place-and-type
    new-type-id
      type-specifier-seq
      new-declarator-opt
        direct-new-declarator
          <type_array>^* || <type_new_array> [cpp/syntax/syntax.act]
            make_array_dim^* [basetype.c] || make_new_array_dim [allocate.c]
  new-initialiser-opt
    expression-list-opt
    <exp_new_init> || <exp_new_none>
      make_new_init [allocate.c], init = 1 || make_new_init, init = 0
  <exp_new>
    make_new_exp [allocate.c]

The function make_new_init [allocate.c] builds the initializer(s) for the
allocated object(s), and the function make_new_exp [allocate.c] is in charge of
processing the 'new' expression. But make_new_exp almost never builds a 'new'
expression; most of the times, it builds a call to the (overloaded) function
'operator new', following the usual steps: resolves the overloading calling
resolve_call [overload.c], marks the identifier 'operator new' as having
been used calling use_func_id [identifier.c], and finally builds the call to
the function 'operator new' with apply_func_id [function.c].

Thus, in the programs whose compilation fails (arraybound.C, arraybound_f.C),
and the program that works by chance (arraybound_e.C), copy_exp [copy.c] and
expand_type [token.c] are trying to instantiate a T-parameterized call to the
function 'operator new[]', but, as I said before, they can not do that.

But there is a case that works genuinely well. It is arraybound_t.C, where the
type of the array elements is just T, the template parameter (a "projection").
Why? Because this case is dealt with using a conditional

    /* Check for template parameters */
    if ( is_templ_type ( t ) ) {
	e = make_templ_new ( u, v, ret, b, place, init ) ;
	return ( e ) ;
    }

inside the function make_new_exp [allocate.c]. What make_templ_new [allocate.c]
does is to build a 'new' expression using the types and expressions gathered
during the parsing. This is the only case in which make_new_exp does what its
name implies. Afterwards, when copy_exp [copy.c] is instantiating the template,
this 'new' expression is processed in the case exp_opn_tag, where the function
apply_nary [operator.c] is called; and then apply_nary reaches the cases
lex_new and lex_new_Hfull, where make_new_exp [allocate.c] is called again to
complete the task that was interrupted by the appearance of the template
parameter T. But now the template parameter T has been expanded by the call to
expand_type [token.c] placed at the beginning of copy_exp [copy.c], and the
application of the function 'operator new[]' is successfully built.

Why does not this conditional work in the cases arraybound.C and
arraybound_f.C? Because, as usual, the function is_templ_type [template.c]
does not do what its name implies: it tells whether its argument is a template
parameter (not a template-dependent type) of the template that is currently
being processed (remember that, the first time, make_new_exp [allocate.c]
is being called from the parser, which is LL(1)). To fix this problem, what we
need is to use the function is_templ_depend [template.c], which tells whether
its argument depends on some parameter of the template that is currently being
processed.

Finally, two subtleties. First, we must move the call to is_templ_depend from
where the call to is_templ_type is now to the beginning of the function
make_new_exp, since this call must be applied to the whole type, array type
included, in order to treat cases like arraybound_f.C. Now, the call to
is_templ_type is being made on the type of the array elements, not on the type
of the 'new' expression. Second, all this analysis is true of the function
make_new_init [allocate.c], which suffers from the same problem as
make_new_exp.

The modifications we have to make to the file allocate.c to fix this bug are:

[ ] Call to is_templ_depend at the beginning of the functions make_new_exp and
    make_new_init and record its result in the boolean variable 'templ',

    int templ = is_templ_depend ( t ) ;

[ ] In the functions make_new_exp and make_new_init, change the conditionals

    if ( is_templ_type ( t ) ) {
    }

    for

    if ( templ ) {
    }

[ ] In the function make_new_exp, change the commment

    /* Check for template parameters */

    for

    /* Check for dependent types */

[ ] In the function make_templ_new, change the comment

    type is a template parameter.  t gives the given type with array

    for

    type is a dependent type.  t gives the given type with array