The InnoDB Forums - Embedded InnoDB

embedded InnoDb on a cluster file system (1 reply)

markd — Tue, 03 Aug 2010 21:50:39 +0300

We have an application where we would like to use an embedded database
on the GPFS cluster file system. Is embedded InnoDb suitable for such
an application? GPFS has a lock manager and support fcntl locking.

Thanks!

bug: ib_cfg_set_text does not documented that string is not copied (no replies)

markd — Mon, 02 Aug 2010 01:15:09 +0300

ib_cfg_set_text does not copy string value, it just saves a pointer. However
this not documented. This can lead to configuration corruption if a dynamic
string is used. It would be far safer if the string was copied, otherwise the
documentation should contain a very obvious warning.

Minor question about InnoDB documentation - #3 (no replies)

underhill — Sun, 01 Aug 2010 15:23:30 +0300

Embedded InnoDB 1.0 User’s Guide and Reference, p. 97:

fast shutdown - A shutdown procedure that is required before installation of the InnoDB Plugin. From the
MySQL command line, issue the following command before performing the shutdown:
SET GLOBAL innodb_fast_shutdown=0;

This seems confusing to me. Don't you want a slow (full) shutdown before installing the plugin? I assume that's what innodb_fast_shutdown=0 does, but wouldn't that be the opposite of a fast shutdown, as stated above?

Minor question about InnoDB documentation - #2 (no replies)

underhill — Sun, 01 Aug 2010 15:18:19 +0300

Embedded InnoDB 1.0 User’s Guide and Reference, p. 38:

Top of the page, Section 6.5, Handles

ib_crsrl_t Handle to an InnoDB cursor.

Is this ib_crsrl_t or ib_crsr_t?

Minor question about InnoDB documentation - #1 (3 replies)

underhill — Tue, 31 Aug 2010 05:38:39 +0300

Embedded InnoDB 1.0 User’s Guide and Reference, p. 26:

ib_err_t err;
int res = ~0;
ib_tpl_t key = NULL;

/* Create a tuple for searching an index. */
key_tpl = ib_sec_search_tuple_create(crsr);

/* Set the value to delete. */
ib_col_set_value(key, 0, "b", strlen("b"));
ib_col_set_value(key, 1, "z", strlen("z"));

On line 5 above, shouldn't "key_tpl" be "key"?

INNODB embedded with .net and c# (1 reply)

magixxfactory — Fri, 23 Jul 2010 21:08:45 +0300

Hello,
is it possible to use .net, c# and/ or ADO.NET with INNODB embededded and if so are there samples available ?
Thanks
Uwe

Embedded InnoDB 1.0.7 (3 replies)

Jonah H. Harris — Thu, 24 Jun 2010 01:01:21 +0300

I began merging the 1.0.7 changes into Embedded InnoDB 1.0.6, but found there are very few actual changes as it looks like the 1.0.6 tree was fairly close to 1.0.7 release. As far as new changes go, the 1.0.7 code appears to contain a new work queue (ut0wqueue.[ch]) that doesn't seem to be used.

The majority of differences are related to the following things:

Is there any preferred way of handling some of the above? Can all the InnoDB/Oracle copyrights just be updated such that the latest one is 2010 rather than a variety of 200X? For the functions with different arguments, is there any objection to just defining something like EMBEDDED_BUILD or PLUGIN_BUILD and using the preprocessor to handle the differences (which are very few)?

Example:

int
dtuple_coll_cmp(
/*============*/
    const dtuple_t* tuple1, /*!< in: tuple 1 */
    const dtuple_t* tuple2) /*!< in: tuple 2 */
{
...
        cmp = cmp_dfield_dfield(cmp_ctx, field1, field2);
...

becomes

int
dtuple_coll_cmp(
/*============*/
#ifndef PLUGIN_BUILD
    void*           cmp_ctx,/*!< in: client compare context */
#endif /* !PLUGIN_BUILD */
    const dtuple_t* tuple1, /*!< in: tuple 1 */
    const dtuple_t* tuple2) /*!< in: tuple 2 */
{
...
#ifdef PLUGIN_BUILD
        cmp = cmp_dfield_dfield(field1, field2);
#else /* Embedded requires usage of a comparison context */
        cmp = cmp_dfield_dfield(cmp_ctx, field1, field2);
#endif /* PLUGIN_BUILD */
...

Thoughts?

Embedded InnoDB is awesome! (2 replies)

Jonah H. Harris — Wed, 02 Jun 2010 11:27:27 +0300

As Heikki and a few others know, several years ago I had pulled InnoDB out of the MySQL code and created a standalone version of it myself. Thankfully, it was a fairly easy task given that InnoDB was originally designed to be a standalone database engine based on many of the Gray and Reuter concepts. Regardless, I'm extremely happy to see InnoDB once again embracing a standalone, API-driven, embedded system approach. I hope many people play with and adopt Embedded InnoDB. To that end, I'll also try to help promote it by writing up a blog entry or two on it sometime in the near future.

In short, given that I've been using quite a few embedded database systems lately (eXtremeDB, Tokyo Cabinet, SQLite, etc.), it's *great* to see InnoDB now available for similar usage without all the MySQL kludge on top of it.

I just started a pretty cool project with Embedded InnoDB this evening; stay tuned for info about it ;-)

Keep up the great work!

-Jonah

DB_MISSING_HISTORY from ib_cursor_first (no replies)

stewart — Wed, 07 Apr 2010 04:15:56 +0300

With the following sequence of events (see main in attached source) you end up hitting the assert for DB_MISSING_HISTORY on the ib_cursor_first instead of getting any error back from ib_cursor_attach_trx that the table didn't exist in your snapshot.

The definition for DB_MISSING_HISTORY in the docs is also a bit misleading in this context. It says:

Required history data has been deleted, due to lack of space in rollback
segment.

when it is more along the lines of "the history never existed".

Not sure what the best thing to do is here... perhaps just an expansion of the explanation of the error code? Or a different one? It seems like that DB_MISSING_HISTORY could be rather a rollback the transaction and tell the user to try again error while "you can't see that table in your snapshot" is more of a temporary one.

cluster search tuple column numbers (no replies)

stewart — Thu, 25 Mar 2010 09:48:26 +0200

I was surprised to learn today that the clustered index search tuples column numbers are the column number of the clustered index, not the column number of the table... while not really a problem, it was a bit of a surprise. Perhaps this could be made a bit more clear in the docs.

cheers,
stewart.

using DB_ROW_ID for handler::position() functionality (no replies)

stewart — Mon, 22 Mar 2010 16:53:02 +0200

It looks as though ib_col_set_ doesn't let you set a value for a system column (such as DB_ROW_ID) to search on. (having said that... not entirely sure how to get the value either).

I was hoping to use this through the embedded innodb api the same way as the innobase ha_innodb.cc does for ::position() and ::rnd_pos() - for getting and setting a cursor on tables without a explicit primary key (i.e. a primary key on rowid).

XA and embedded innodb? (no replies)

stewart — Wed, 10 Mar 2010 04:06:06 +0200

It looks like it's possible to get a transaction state that is IB_TRX_PREPARED from ib_trx_state() but I don't see any way to start a XA transaction or any function like ib_trx_prepare(). Is this planned to exist?

bug: ib_cursor_truncate commits transaction but doesn't release schema lock (no replies)

stewart — Wed, 10 Mar 2010 04:00:27 +0200

I've found that ib_cursor_truncate currently commits the transaction on success.

from api0api.c:
/* This function currently commits the transaction
on success. */
err = ddl_truncate_table(table, trx);

which isn't documented (except in the source :)

BUT, it doesn't release the schema lock... so unlike other DDL ops on success you have to release the schema lock yourself.

it'd be great if ib_cursor_truncate() was a bit more consistent with the rest of the DDL api and the user called commit (which released the schema lock).

Docs bug: ib_cusor_truncate needs schema lock (no replies)

stewart — Wed, 10 Mar 2010 03:52:05 +0200

I've found that ib_cursor_truncate() bombs out if you don't have the exclusive schema lock. This doesn't seem to be explicitly documented (but does seem a bit obvious after a bit of thought).

I also assume that ib_cursor_truncate() is the same as truncate table in the innodb plugin, in that it's not transactional so it would be suitable for TRUNCATE TABLE and not DELETE FROM t;

endian safety of innodb tablespaces (1 reply)

stewart — Fri, 26 Feb 2010 07:56:50 +0200

Assuming I take care of the safety of what i'm storing with ib_col_set_value.... are the tablespace files (ibd) themselves endian safe? Should I expect to be able to take one created on little endian and read it on big endian?

Embedded InnoDB 1.0.6.6750 released (no replies)

Vasil Dimov — Thu, 25 Feb 2010 17:03:28 +0200

The next version of Embedded InnoDB 1.0.6.6750 has just been released!

This long-awaited version is based on the InnoDB Plugin 1.0.6, see the full announcement on the InnoDB web site.

Download Embedded InnoDB 1.0.6.6750.

Thank you, and enjoy!

Finding the cause of DB_DUPLICATE_KEY (2 replies)

stewart — Thu, 25 Feb 2010 14:27:07 +0200

i have code for row insertion that looks a bit like this:

err= ib_cursor_insert_row(cursor, tuple);

if (err == DB_DUPLICATE_KEY)
ret= HA_ERR_FOUND_DUPP_KEY;

However I have not found any immediately obvious way to then find out from embedded innodb which key was the cause of the duplicate key error.

it'd be great to get back key id and key name.

cheers,
stewart

ib_cursor_set_match_mode(cursor, IB_EXACT_MATCH) not working? (2 replies)

stewart — Sat, 13 Feb 2010 09:05:59 +0200

To do an exact lookup on clustered index, i tried:

ib_cursor_set_match_mode(cursor, IB_EXACT_MATCH);

err= ib_cursor_moveto(cursor, search_tuple, IB_CUR_GE, &res);

but this always came back with DB_RECORD_NOT_FOUND.

if i remove the set_match_mode and check 'res==0' then it works.

am I missing something?

docs bug: ib_create_database() returns ib_bool_t not ib_err_t (2 replies)

stewart — Sat, 13 Feb 2010 04:41:17 +0200

although i think ib_err_t could be ideal. It'd be great to get the real errno back.

Cant do clustered index search on SYS_TABLES? (2 replies)

stewart — Fri, 12 Feb 2010 08:26:48 +0200

I was trying to do a prefix search on SYS_TABLES.

That is, looking for table names with "test/" as a prefix.

However, when I set up the search tuple, ib_col_set_value() dies in an assert in api0api.c:4058

which is in the switch for data type (the assert is in the default: bit of the switch)

Interestingly, tests/ib_dict prints this:
CREATE TABLE SYS_TABLES(
NAME UNKNOWN,
ID UNKNOWN,
N_COLS INT,
TYPE INT,
MIX_ID UNKNOWN,
MIX_LEN INT,
CLUSTER_NAME UNKNOWN,
SPACE INT,
CREATE UNIQUE INDEX CLUST_IND ON SYS_TABLES(NAME);
CREATE INDEX ID_IND ON SYS_TABLES(ID);

which seems to indicate that something special is going on.

docs bug in ib_schema_tables_iterator (2 replies)

stewart — Fri, 12 Feb 2010 08:05:05 +0200

on page 50 of the docs:

6.7.6.3. ib_schema_tables_iterate
Synopsis
ib_err_t ib_table_schema_add_index(
ib_trx_t ib_trx,
ib_schema_visitor_table_all_t visitor,
void* arg);

as you can see, the function names don't match :)

Suggestion: warn_unused_result on ib_err_t returning functions (2 replies)

stewart — Fri, 12 Feb 2010 08:10:35 +0200

It'd be neat if we had warn_unused_result on all functions returning ib_err_t. i.e. warn if a programs calls the function but doesn't check the return. recent glibc does this for calls like write(2) and it does help find bugs.

I'd happily provide a patch... but at least innodb.h says it's a generated file. Where is it generated from?

from gcc manual:

The `warn_unused_result' attribute causes a warning to be emitted
if a caller of the function with this attribute does not use its
return value. This is useful for functions where not checking the
result is either a security problem or always a bug, such as
`realloc'.

int fn () __attribute__ ((warn_unused_result));
int foo ()
{
if (fn () < 0) return -1;
fn ();
return 0;
}

results in warning on line 5.

ib_table_drop takes schema lock (7 replies)

stewart — Fri, 12 Feb 2010 08:13:02 +0200

I was doing something like this:

innodb_schema_transaction= ib_trx_begin(IB_TRX_REPEATABLE_READ);

innodb_err= ib_schema_lock_exclusive(innodb_schema_transaction);

if (innodb_err == DB_SUCCESS)
innodb_err= ib_table_drop(table_name.c_str());

and I was getting a deadlock in ib_table_drop() with the following backtrace:
#0 ib_schema_lock_exclusive (ib_trx=0xfbf580) at api/api0api.c:5293
#1 0x00007ffff48981ef in ib_table_drop (name=0xfa16f8 "./test/t1")
at api/api0api.c:2611
#2 0x00007ffff491b95e in EmbeddedInnoDBEngine::doDropTable (this=0xf92980,
session=..., table_name=...)
at plugin/embedded_innodb/embedded_innodb_engine.cc:265

This seems to be inconsistent with other DDL operations (or at least undocumented behavior).

Conflicting information on ib_schema_unlock() (1 reply)

stewart — Thu, 11 Feb 2010 09:08:59 +0200

page 1 of the docs say:

"The data dictionary must be locked in exclusive mode when creating tables using the API func-
tion ib_schema_lock_exclusive(). The schema latches must be release after committing the DDL transaction using
ib_schema_unlock()."

While the reference at the back says:
"Currently, when a transaction is rolled back by InnoDB, it doesn’t release the data dictionary lock. This must be done explicitly by the application."

and none of the examples ever call ib_schema_unlock().

Do we only need to unlock on rollback of DDL txn? Do we need to unlock after commit of DDL txn?

and should it be before or after the rollback?

I'd prefer the API to be consistent in either always requiring an explicit unlock or never requiring one.

ib_schema_visitor_t NULL values (1 reply)

stewart — Thu, 11 Feb 2010 03:14:06 +0200

It could be useful to be able to have some functions be NULL in ib_schema_visitor_t when iterating over datadict. e.g. you just want table names, just want indexes or whatever.

currently you end up SIGSEGV with null dereferences, which isn't ideal :)

(or docs to be fixed up to insist that all must be non-null)

ib_table_schema_set_temp_dir and temp tables? (4 replies)

stewart — Tue, 09 Feb 2010 03:39:20 +0200

Does this mean the table won't be logged?

An InnoDB temporary table that isn't logged and is gone on startup could be quite useful during query execution...

Iterating through config and status variables (1 reply)

stewart — Tue, 09 Feb 2010 03:02:54 +0200

This is a feature request... but it'd be neat...

In order to produce a "show innodb status" type output (but better, and in tables), it'd be great if there was a way to get a list of all the configuration and status variables (and their type) from embedded innodb.

This way, if the library API doesn't change but a status or config variable is added, I don't have to modify my code to display it.

Associating a blob with a data ditionary object (4 replies)

stewart — Tue, 09 Feb 2010 03:43:59 +0200

Would it be possible to (at least in future) attach a BLOB to tables in the data dictionary?

Or... is it possible to do transactions on other tables as part of a DDL transaction?

I'm wanting to store the table definition protobuf message (equiv of FRM) inside InnoDB. This closes a long standing crash safety bug.

ib_schema_lock_shared ? (6 replies)

stewart — Fri, 12 Feb 2010 07:56:22 +0200

What does this do? In what contexts could I use it? I gather DDL under it is not a good idea?

[BUG] Embedded InnoDB and innodb plugin cannot be simultaneously loaded (3 replies)

stewart — Tue, 09 Feb 2010 02:12:16 +0200

In trying to have a plugin which uses Embedded InnoDB in a server that also has the InnoDB plugin loaded, on ib_init() for embedded innodb, I get SIGSEGV with the following backtrace:

#1 0x00007fffe4ecf95c in ut_print_timestamp (ib_stream=0x0) at ut/ut0ut.c:239
#2 0x00007fffe4ece58a in ut_dbg_assertion_failed (
expr=0x7fffe4edea10 "!srv_was_started",
file=0x7fffe4edfaa6 "%02d%02d%02d %2d:%02d:%02d", line=10)
at ut/ut0dbg.c:59
#3 0x00007fffe4ecf310 in ut_mem_init () at ut/ut0mem.c:93
#4 0x00007fffe4ed1469 in ib_init () at api/api0api.c:727
#5 0x00007fffe4f545b7 in embedded_innodb_init (registry=...)
at plugin/embedded_innodb/embedded_innodb_engine.cc:241
#6 0x0000000000512f2c in plugin_initialize (registry=..., module=0x1394910)
at drizzled/plugin/loader.cc:345
#7 0x0000000000513483 in plugin_init (registry=..., argc=0xb545d0,
argv=0xb8b310, skip_init=false) at drizzled/plugin/loader.cc:478
#8 0x0000000000411878 in init_server_components (plugins=...)
at drizzled/drizzled.cc:1278
#9 0x00000000004137a5 in main (argc=22, argv=0x7fffffffd418)
at drizzled/drizzled.cc:2360

I assume this is because having the two loaded at once won't agree at all (about to check with disabling the default innodb plugin)