Repository navigation
Expand file tree
/
Copy pathREADME
More file actions
3604 lines (2877 loc) · 171 KB
/
Copy pathREADME
File metadata and controls
3604 lines (2877 loc) · 171 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
NAME
Ora2Pg - Oracle to PostgreSQL database schema converter
DESCRIPTION
Ora2Pg is a free tool used to migrate an Oracle database to a PostgreSQL
compatible schema. It connects to your Oracle database, scans it
automatically and extracts its structure or data, then generates SQL
scripts that you can load into your PostgreSQL database.
Ora2Pg can be used for anything from reverse engineering an Oracle
database to huge enterprise database migration or simply replicating
some Oracle data into a PostgreSQL database. It is really easy to use
and doesn't require any Oracle database knowledge other than providing
the parameters needed to connect to the Oracle database.
FEATURES
Ora2Pg consists of a Perl script (ora2pg) and a Perl module (Ora2Pg.pm).
The only thing you have to modify is the configuration file ora2pg.conf
by setting the DSN to the Oracle database and optionally the name of a
schema. Once that's done, you just have to set the type of export you
want: TABLE with constraints, VIEW, MVIEW, TABLESPACE, SEQUENCE,
INDEXES, TRIGGER, GRANT, FUNCTION, PROCEDURE, PACKAGE, PARTITION, TYPE,
INSERT or COPY, FDW, QUERY, KETTLE, SYNONYM.
By default, Ora2Pg exports to a file that you can load into PostgreSQL
with the psql client, but you can also import directly into a PostgreSQL
database by setting its DSN into the configuration file. With all
configuration options of ora2pg.conf, you have full control of what
should be exported and how.
Features included:
- Export full database schema (tables, views, sequences, indexes), with
unique, primary, foreign key and check constraints.
- Export grants/privileges for users and groups.
- Export range/list partitions and sub partitions.
- Export a table selection (by specifying the table names).
- Export Oracle schema to a PostgreSQL 8.4+ schema.
- Export predefined functions, triggers, procedures, packages and
package bodies.
- Export full data or following a WHERE clause.
- Full support of Oracle BLOB object as PG BYTEA.
- Export Oracle views as PG tables.
- Export Oracle user defined types.
- Provide some basic automatic conversion of PLSQL code to PLPGSQL.
- Works on any platform.
- Export Oracle tables as foreign data wrapper tables.
- Export materialized view.
- Show a report of an Oracle database content.
- Migration cost assessment of an Oracle database.
- Migration difficulty level assessment of an Oracle database.
- Migration cost assessment of PL/SQL code from a file.
- Migration cost assessment of Oracle SQL queries stored in a file.
- Generate XML ktr files to be used with Pentaho Data Integrator (Kettle)
- Export Oracle locator and spatial geometries into PostGIS.
- Export DBLINK as Oracle FDW.
- Export SYNONYMS as views.
- Export DIRECTORY as external table or directory for external_file extension.
- Dispatch a list of SQL orders over multiple PostgreSQL connections
- Perform a diff between Oracle and PostgreSQL database for test purposes.
- MySQL/MariaDB and Microsoft SQL Server migration.
Ora2Pg does its best to automatically convert your Oracle database to
PostgreSQL but there's still manual work to do. The Oracle specific
PL/SQL code generated for functions, procedures, packages and triggers
has to be reviewed to match the PostgreSQL syntax. You will find some
useful recommendations on porting Oracle PL/SQL code to PostgreSQL
PL/PGSQL at "Converting from other Databases to PostgreSQL", section:
Oracle (http://wiki.postgresql.org/wiki/Main_Page).
See http://ora2pg.darold.net/report.html for an HTML sample of an Oracle
database migration report.
INSTALLATION
All Perl modules can always be found at CPAN (http://search.cpan.org/).
Just type the full name of the module (ex: DBD::Oracle) into the search
input box, it will bring you to the page for download.
Releases of Ora2Pg are published at SF.net
(https://sourceforge.net/projects/ora2pg/).
On Windows(TM) you should install Strawberry Perl
(http://strawberryperl.com/) and the OSes corresponding Oracle clients.
Since version 5.32, the Perl distribution includes pre-compiled driver
for DBD::Oracle and DBD::Pg.
Required
The Oracle Instant Client or a full Oracle installation must be
installed on the system. You can download the RPM from Oracle download
center:
rpm -ivh oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm
rpm -ivh oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm
rpm -ivh oracle-instantclient12.2-jdbc-12.2.0.1.0-1.x86_64.rpm
rpm -ivh oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm
or simply download the corresponding ZIP archives from Oracle download
center and install them where you want, for example:
/opt/oracle/instantclient_12_2/
You also need a modern Perl distribution (Perl 5.10 or later). To
connect to a database and proceed with its migration, you need the DBI
Perl module > 1.614. To migrate an Oracle database, you need the
DBD::Oracle Perl module to be installed.
To install DBD::Oracle and have it working, you need to have the Oracle
client libraries installed and the ORACLE_HOME environment variable must
be defined.
If you plan to export a MySQL database, you need to install the Perl
module DBD::MySQL which requires that the MySQL client libraries are
installed.
If you plan to export a SQL Server database, you need to install the
Perl module DBD::ODBC which requires that the unixODBC package is
installed.
On some Perl distributions, you may need to install the Time::HiRes Perl
module.
If your distribution doesn't include these Perl modules, you can install
them using CPAN:
perl -MCPAN -e 'install DBD::Oracle'
perl -MCPAN -e 'install DBD::MySQL'
perl -MCPAN -e 'install DBD::ODBC'
perl -MCPAN -e 'install Time::HiRes'
otherwise, use the packages provided by your distribution.
Optional
By default, Ora2Pg dumps exports to flat files. To load them into your
PostgreSQL database, you need the PostgreSQL client (psql). If you don't
have it on the host running Ora2Pg, you can always transfer these files
to a host with the psql client installed. If you prefer to load exports
'on the fly', the Perl module DBD::Pg is required.
Ora2Pg allows you to dump all output in a compressed gzip file. To do
this, you need the Compress::Zlib Perl module or, if you prefer using
bzip2 compression, the program bzip2 must be available in your PATH.
If your distribution doesn't include these Perl modules, you can install
them using CPAN:
perl -MCPAN -e 'install DBD::Pg'
perl -MCPAN -e 'install Compress::Zlib'
otherwise, use the packages provided by your distribution.
Instruction for SQL Server
For SQL Server, you need to install the unixodbc package and the Perl
DBD::ODBC driver:
sudo apt install unixodbc
sudo apt install libdbd-odbc-perl
or
sudo yum install unixodbc
sudo yum install perl-DBD-ODBC
sudo yum install perl-DBD-Pg
Then install the Microsoft ODBC Driver for SQL Server. Follow the
instructions for to your operating system from here:
https://docs.microsoft.com/fr-fr/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver16
Once done, set the following in the /etc/odbcinst.ini file by adjusting
the SQL Server ODBC driver version:
[msodbcsql18]
Description=Microsoft ODBC Driver 18 for SQL Server
Driver=/opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.0.so.1.1
UsageCount=1
See ORACLE_DSN to learn how to use the driver to connect to your MSSQL
database.
Installing Ora2Pg
Like any other Perl Module, Ora2Pg can be installed with the following
commands:
tar xjf ora2pg-x.x.tar.bz2
cd ora2pg-x.x/
perl Makefile.PL
make && make install
This will install Ora2Pg.pm into your site Perl repository, ora2pg into
/usr/local/bin/ and ora2pg.conf into /etc/ora2pg/.
On Windows(TM), you may use instead:
perl Makefile.PL
gmake && gmake install
This will install scripts and libraries into your Perl site installation
directory and the ora2pg.conf file as well as all documentation files
into C:\ora2pg\
To install ora2pg in a different directory than the default one, simply
use this command:
perl Makefile.PL PREFIX=<your_install_dir>
make && make install
then set PERL5LIB to the path to your installation directory before
using Ora2Pg.
export PERL5LIB=<your_install_dir>
ora2pg -c config/ora2pg.conf -t TABLE -b outdir/
Packaging
If you want to build binary packages for your preferred Linux
distribution, take a look at the packaging/ directory of the source
tarball. It contains everything needed to build RPM, Slackware and
Debian packages. See the README file in that directory.
Installing DBD::Oracle
Ora2Pg needs the Perl module DBD::Oracle for connectivity to an Oracle
database from Perl DBI. You can get DBD::Oracle from CPAN, a Perl module
repository.
After setting ORACLE_HOME and LD_LIBRARY_PATH environment variables as
root user, install DBD::Oracle. Proceed as follows:
export LD_LIBRARY_PATH=/usr/lib/oracle/12.2/client64/lib
export ORACLE_HOME=/usr/lib/oracle/12.2/client64
perl -MCPAN -e 'install DBD::Oracle'
If you are running for the first time, it will ask many questions; you
can keep defaults by pressing ENTER key, but you need to provide one
appropriate mirror site for CPAN to download the modules. Install
through CPAN manually if the above doesn't work:
#perl -MCPAN -e shell
cpan> get DBD::Oracle
cpan> quit
cd ~/.cpan/build/DBD-Oracle*
export LD_LIBRARY_PATH=/usr/lib/oracle/11.2/client64/lib
export ORACLE_HOME=/usr/lib/oracle/11.2/client64
perl Makefile.PL
make
make install
Installing DBD::Oracle requires that the three Oracle packages:
instant-client, SDK and SQLplus are installed as well as the libaio1
library.
If you are using Instant Client from ZIP archives, the LD_LIBRARY_PATH
and ORACLE_HOME will be the same and must be set to the directory where
you have installed the files. For example:
/opt/oracle/instantclient_12_2/
CONFIGURATION
Configuring Ora2Pg can be as simple as choosing the Oracle database to
export and choosing the export type. This can be done in a minute.
By reading this documentation you will also be able to:
- Select only certain tables and/or columns for export.
- Rename some tables and/or columns during export.
- Select data to export following a WHERE clause per table.
- Delay database constraints during data loading.
- Compress exported data to save disk space.
- and much more.
The Oracle database migration is fully controlled through a single
configuration file named ora2pg.conf. The format of this file consists
of a directive name in upper case followed by a tab character and a
value. Comments are lines beginning with a #.
There's no specific order to place the configuration directives, they
are set at the time they are read in the configuration file.
For configuration directives that just take a single value, you can use
them multiple times in the configuration file but only the last
occurrence found in the file will be used. For configuration directives
that allow a list of values, you can use them multiple times, the values
will be appended to the list. If you use the IMPORT directive to load a
custom configuration file, directives defined in this file will be
stored from the place the IMPORT directive is found, so it is better to
put it at the end of the configuration file.
Values set in command line options will override values from the
configuration file.
Ora2Pg usage
First of all be sure that libraries and binaries paths include the
Oracle Instant Client installation:
export LD_LIBRARY_PATH=/usr/lib/oracle/11.2/client64/lib
export PATH="/usr/lib/oracle/11.2/client64/bin:$PATH"
By default, Ora2Pg will look for /etc/ora2pg/ora2pg.conf configuration
file. If the file exists, you can simply execute:
/usr/local/bin/ora2pg
or under Windows(TM) run ora2pg.bat file, located in your Perl bin
directory. Windows(TM) users may also find a template configuration file
in C:\ora2pg
If you want to call another configuration file, just give the path as a
command line argument:
/usr/local/bin/ora2pg -c /etc/ora2pg/new_ora2pg.conf
Here are all command line parameters available when using ora2pg:
Usage: ora2pg [-dhpqv --estimate_cost --dump_as_html] [--option value]
-a | --allow str : Comma separated list of objects to allow from export.
Can be used with SHOW_COLUMN too.
-b | --basedir dir: Set the default output directory, where files
resulting from exports will be stored.
-c | --conf file : Set an alternate configuration file other than the
default /etc/ora2pg/ora2pg.conf.
-C | --cdc_file file: File used to store/read SCN per table during export.
default: TABLES_SCN.log in the current directory. This
is the file written by the --cdc_ready option.
-d | --debug : Enable verbose output.
-D | --data_type str : Allow custom type replacement at command line.
-e | --exclude str: Comma separated list of objects to exclude from export.
Can be used with SHOW_COLUMN too.
-h | --help : Print this short help.
-g | --grant_object type : Extract privilege from the given object type.
See possible values with GRANT_OBJECT configuration.
-i | --input file : File containing Oracle PL/SQL code to convert with
no Oracle database connection initiated.
-j | --jobs num : Number of parallel process to send data to PostgreSQL.
-J | --copies num : Number of parallel connections to extract data from Oracle.
-l | --log file : Set a log file. Default is stdout.
-L | --limit num : Number of tuples extracted from Oracle and stored in
memory before writing, default: 10000.
-m | --mysql : Export a MySQL database instead of an Oracle schema.
-M | --mssql : Export a Microsoft SQL Server database.
-n | --namespace schema : Set the Oracle schema to extract from.
-N | --pg_schema schema : Set PostgreSQL's search_path.
-o | --out file : Set the path to the output file where SQL will
be written. Default: output.sql in running directory.
-O | --options : Used to override any configuration parameter, it can
be used multiple time. Syntax: -O "PARAM_NAME=value"
-p | --plsql : Enable PLSQL to PLPGSQL code conversion.
-P | --parallel num: Number of parallel tables to extract at the same time.
-q | --quiet : Disable progress bar.
-r | --relative : use \ir instead of \i in the psql scripts generated.
-s | --source DSN : Allow to set the Oracle DBI datasource.
-S | --scn SCN : Allow to set the Oracle System Change Number (SCN) to
use to export data. It will be used in the WHERE clause
to get the data. It is used with action COPY or INSERT.
-t | --type export: Set the export type. It will override the one
given in the configuration file (TYPE).
-T | --temp_dir dir: Set a distinct temporary directory when two
or more ora2pg are run in parallel.
-u | --user name : Set the Oracle database connection user.
ORA2PG_USER environment variable can be used instead.
-v | --version : Show Ora2Pg Version and exit.
-w | --password pwd : Set the password of the Oracle database user.
ORA2PG_PASSWD environment variable can be used instead.
-W | --where clause : Set the WHERE clause to apply to the Oracle query to
retrieve data. Can be used multiple time.
--forceowner : Force ora2pg to set tables and sequences owner like in
Oracle database. If the value is set to a username this one
will be used as the objects owner. By default it's the user
used to connect to the Pg database that will be the owner.
--nls_lang code: Set the Oracle NLS_LANG client encoding.
--client_encoding code: Set the PostgreSQL client encoding.
--view_as_table str: Comma separated list of views to export as table.
--estimate_cost : Activate the migration cost evaluation with SHOW_REPORT
--cost_unit_value minutes: Number of minutes for a cost evaluation unit.
default: 5 minutes, corresponds to a migration conducted by a
PostgreSQL expert. Set it to 10 if this is your first migration.
--dump_as_html : Force ora2pg to dump report in HTML, used only with
SHOW_REPORT. Default is to dump report as simple text.
--dump_as_csv : As above but force ora2pg to dump report in CSV.
--dump_as_json : As above but force ora2pg to dump report in JSON.
--dump_as_sheet : Report migration assessment with one CSV line per database.
--init_project name: Initialise a typical ora2pg project tree. Top directory
dump_as_* selected switches, suffixes
will be .html, .csv, .json.
--init_project name: Initialise a typical ora2pg project tree. Top directory
will be created under project base dir.
--project_base dir : Define the base dir for ora2pg project trees. Default
is current directory.
--print_header : Used with --dump_as_sheet to print the CSV header
especially for the first run of ora2pg.
--human_days_limit num : Set the number of person-days limit where the migration
assessment level switch from B to C. Default is set to
5 person-days.
--audit_user list : Comma separated list of usernames to filter queries in
the DBA_AUDIT_TRAIL table. Used only with SHOW_REPORT
and QUERY export type.
--pg_dsn DSN : Set the datasource to PostgreSQL for direct import.
--pg_user name : Set the PostgreSQL user to use.
--pg_pwd password : Set the PostgreSQL password to use.
--count_rows : Force ora2pg to perform a real row count in TEST,
TEST_COUNT and SHOW_TABLE actions.
--no_header : Do not append Ora2Pg header to output file
--oracle_speed : Use to know at which speed Oracle is able to send
data. No data will be processed or written.
--ora2pg_speed : Use to know at which speed Ora2Pg is able to send
transformed data. Nothing will be written.
--blob_to_lo : export BLOB as large objects, can only be used with
action SHOW_COLUMN, TABLE and INSERT.
--cdc_ready : use current SCN per table to export data and register
them into a file named TABLES_SCN.log per default. It
can be changed using -C | --cdc_file.
--lo_import : use psql \lo_import command to import BLOB as large
object. Can be use to import data with COPY and import
large object manually in a second pass. It is recquired
for BLOB > 1GB. See documentation for more explanation.
--mview_as_table str: Comma separated list of materialized views to export
as regular table.
--drop_if_exists : Drop the object before creation if it exists.
--delete clause : Set the DELETE clause to apply to the Oracle query to
be applied before importing data. Can be used multiple
time.
--oracle_fdw_prefetch: Set the oracle_fdw prefetch value. Larger values
generally result in faster data transfer at the cost
of greater memory utilisation at the destination.
See full documentation at https://ora2pg.darold.net/ for more help or
see manpage with 'man ora2pg'.
ora2pg will return 0 on success, 1 on error. It will return 2 when a
child process has been interrupted and you've gotten the warning
message: "WARNING: an error occured during data export. Please check
what's happened." Most of the time this is an OOM issue, so first try
reducing DATA_LIMIT value.
For developers, it is possible to add your own custom option(s) in the
Perl script ora2pg as any configuration directive from ora2pg.conf can
be passed in lower case to the new Ora2Pg object instance. See ora2pg
code on how to add your own option.
Note that performance might be improved by updating stats on Oracle:
BEGIN
DBMS_STATS.GATHER_SCHEMA_STATS
DBMS_STATS.GATHER_DATABASE_STATS
DBMS_STATS.GATHER_DICTIONARY_STATS
END;
Generate a migration template
The two options --project_base and --init_project indicate to ora2pg
that it should create a project template with a work tree, a
configuration file and a script to export all objects from the Oracle
database. Here is a sample of the command usage:
ora2pg --project_base /app/migration/ --init_project test_project
Creating project test_project.
/app/migration/test_project/
schema/
dblinks/
directories/
functions/
grants/
mviews/
packages/
partitions/
procedures/
sequences/
synonyms/
tables/
tablespaces/
triggers/
types/
views/
sources/
functions/
mviews/
packages/
partitions/
procedures/
triggers/
types/
views/
data/
config/
reports/
Generating generic configuration file
Creating script export_schema.sh to automate all exports.
Creating script import_all.sh to automate all imports.
It creates a generic config file where you just have to define the
Oracle database connection and a shell script called export_schema.sh.
The sources/ directory will contain the Oracle code, the schema/
directory will contain the code ported to PostgreSQL. The reports/
directory will contain the HTML and JSON reports with the migration cost
assessment.
If you want to use your own default config file, use the -c option to
give the path to that file. Rename it with .dist suffix if you want
ora2pg to apply the generic configuration values; otherwise, the
configuration file will be copied untouched.
Once you have set the connection to the Oracle Database you can execute
the script export_schema.sh that will export all object types from your
Oracle database and output DDL files into the schema's subdirectories.
At end of the export it will give you the command to export data later
when the import of the schema is done and verified.
You can choose to load the DDL files generated manually or use the
second script import_all.sh to import those files interactively. If this
kind of migration is not something common for you, it's recommended that
you use those scripts.
Oracle database connection
There are 5 configuration directives to control the access to the Oracle
database.
ORACLE_HOME
Used to set the ORACLE_HOME environment variable for the Oracle
libraries required by the DBD::Oracle Perl module.
ORACLE_DSN
This directive is used to set the data source name in the standard
DBI DSN form. For example:
dbi:Oracle:host=oradb_host.myhost.com;sid=DB_SID;port=1521
or
dbi:Oracle:DB_SID
On 18c this could be for example:
dbi:Oracle:host=192.168.1.29;service_name=pdb1;port=1521
For the second notation, the SID should be declared in the
well-known file $ORACLE_HOME/network/admin/tnsnames.ora or in the
path given to the TNS_ADMIN environment variable.
For MySQL the DSN will look like this:
dbi:mysql:host=192.168.1.10;database=sakila;port=3306
The 'sid' part is replaced by 'database'.
For MS SQL Server it will look like this:
dbi:ODBC:driver=msodbcsql18;server=mydb.database.windows.net;database=testdb;TrustServerCertificate=yes
ORACLE_USER and ORACLE_PWD
These two directives are used to define the user and password for
the Oracle database connection. Note that if possible, it is better
to login as Oracle super admin to avoid grant problems during the
database scan and ensure nothing is missing.
If you do not supply credentials with ORACLE_PWD and you have
installed the Term::ReadKey Perl module, Ora2Pg will ask for the
password interactively. If ORACLE_USER is not set it will also be
asked interactively.
To connect to a local ORACLE instance with connections "as sysdba"
you have to set ORACLE_USER to "/" and an empty password.
To make a connection using an Oracle Secure External Password Store
(SEPS), first configure the Oracle Wallet and then set both the
ORACLE_USER and ORACLE_PWD directives to the special value of
"__SEPS__" (without the quotes but with the double underscore).
USER_GRANTS
Set this directive to 1 if you connect to Oracle database as a
simple user and do not have enough grants to extract things from the
DBA_... tables. It will use ALL_... tables instead.
Warning: if you use export type GRANT, you must set this
configuration option to 0 or it will not work.
TRANSACTION
This directive may be used if you want to change the default
isolation level of the data export transaction. Default is now to
set the level to a serializable transaction to ensure data
consistency. The allowed values for this directive are:
readonly: 'SET TRANSACTION READ ONLY',
readwrite: 'SET TRANSACTION READ WRITE',
serializable: 'SET TRANSACTION ISOLATION LEVEL SERIALIZABLE'
committed: 'SET TRANSACTION ISOLATION LEVEL READ COMMITTED',
Releases before 6.2 used to set the isolation level to READ ONLY
transaction but in some cases this was breaking data consistency so
now default is set to SERIALIZABLE.
INPUT_FILE
This directive does not control the Oracle database connection but
rather it purely disables the use of any Oracle database by
accepting a file as argument. Set this directive to a file
containing PL/SQL Oracle Code like function, procedure or full
package body to prevent Ora2Pg from connecting to an Oracle database
and just apply its conversion tool to the content of the file. This
can be used with most export types: TABLE, TRIGGER, PROCEDURE, VIEW,
FUNCTION or PACKAGE, etc.
ORA_INITIAL_COMMAND
This directive can be used to send an initial command to Oracle,
just after the connection. For example to unlock a policy before
reading objects or to set some session parameters. This directive
can be used multiple times.
Data encryption with Oracle server
If your Oracle Client config file already includes the encryption
method, then DBD::Oracle uses those settings to encrypt the connection
while extracting data. For example, if you have configured the Oracle
Client config file (sqlnet.ora or .sqlnet) with the following
information:
# Configure encryption of connections to Oracle
SQLNET.ENCRYPTION_CLIENT = required
SQLNET.ENCRYPTION_TYPES_CLIENT = (AES256, RC4_256)
SQLNET.CRYPTO_SEED = 'should be 10-70 random characters'
Any tool that uses the Oracle client to communicate with the database
will have encrypted connections if you setup session encryption as shown
above.
For example, Perl's DBI uses DBD::Oracle, which uses the Oracle client
for actual database communication. If the Oracle client installation
used by Perl is setup to request encrypted connections, then your Perl
connection to an Oracle database will also be encrypted.
Full details at
https://kb.berkeley.edu/jivekb/entry.jspa?externalID=1005
Testing connection
Once you have set the Oracle database DSN, you can execute ora2pg to see
if it works:
ora2pg -t SHOW_VERSION -c config/ora2pg.conf
will show the Oracle database server version. Take some time here to
test your installation as most problems occur here. The other
configuration steps are more technical.
Troubleshooting
If the output.sql file hasn't exported anything other than the
PostgreSQL transaction header and footer, there are two possible
reasons: 1) The perl script ora2pg dumps an ORA-XXX error, which means
that your DSN or login information is wrong - check the error and your
settings and try again. 2) The perl script says nothing and the output
file is empty: the user lacks permissions to extract something from the
database. Try to connect to Oracle as super user or review the
USER_GRANTS directive above and the next section, especially the SCHEMA
directive.
LOGFILE
By default, all messages are sent to the standard output. If you
provide a file path to this directive, all output will be appended
to this file.
Oracle schema to export
The Oracle database export can be limited to a specific Schema or
Namespace; this may be mandatory depending on the database connection
user.
SCHEMA
This directive is used to set the schema name to use during export.
For example:
SCHEMA APPS
will extract objects associated with the APPS schema.
When no schema name is provided and EXPORT_SCHEMA is enabled, Ora2Pg
will export all objects from all schemas of the Oracle instance with
their names prefixed with the schema name.
EXPORT_SCHEMA
By default, the Oracle schema is not exported into the PostgreSQL
database and all objects are created under the default Pg namespace.
If you want to also export this schema and create all objects under
this namespace, set the EXPORT_SCHEMA directive to 1. This will set
the schema search_path at the top of the export SQL file to the
schema name set in the SCHEMA directive with the default pg_catalog
schema. If you want to change this path, use the directive
PG_SCHEMA.
CREATE_SCHEMA
Enable/disable the CREATE SCHEMA SQL order at the start of the
output file. It is enabled by default and concerns the TABLE export
type.
COMPILE_SCHEMA
By default, Ora2Pg will only export valid PL/SQL code. You can force
Oracle to compile again the invalidated code to get a chance to have
it obtain the valid status and then be able to export it.
Enable this directive to force Oracle to compile schema before
exporting code. When this directive is enabled and SCHEMA is set to
a specific schema name, only invalid objects in this schema will be
recompiled. If SCHEMA is not set then all schema will be recompiled.
To force recompilation of invalid object in a specific schema, set
COMPILE_SCHEMA to the schema name you want to recompile.
This will ask Oracle to validate the PL/SQL that could have been
invalidated after an export/import for example. The 'VALID' or
'INVALID' status applies to functions, procedures, packages and user
defined types. It also concerns disabled triggers.
EXPORT_INVALID
If the above configuration directive is not enough to validate your
PL/SQL code, enable this configuration directive to allow export of
all PL/SQL code even if it is marked as invalid. The 'VALID' or
'INVALID' status applies to functions, procedures, packages,
triggers and user defined types.
PG_SCHEMA
Allows you to define/force the PostgreSQL schema to use. By default,
if you set EXPORT_SCHEMA to 1, the PostgreSQL search_path will be
set to the schema name exported set as value of the SCHEMA
directive.
The value can be a comma-delimited list of schema names but not when
using TABLE export type because in this case it will generate the
CREATE SCHEMA statement and it doesn't support multiple schema
names. For example, if you set PG_SCHEMA to something like
"user_schema, public", the search path will be set like this:
SET search_path = user_schema, public;
forcing the use of an other schema (here user_schema) than the one
from Oracle schema set in the SCHEMA directive.
You can also set the default search_path for the PostgreSQL user you
are using to connect to the destination database by using:
ALTER ROLE username SET search_path TO user_schema, public;
in this case you don't have to set PG_SCHEMA.
SYSUSERS
Without explicit schema, Ora2Pg will export all objects that do not
belong to system schemas or roles:
SYSTEM,CTXSYS,DBSNMP,EXFSYS,LBACSYS,MDSYS,MGMT_VIEW,
OLAPSYS,ORDDATA,OWBSYS,ORDPLUGINS,ORDSYS,OUTLN,
SI_INFORMTN_SCHEMA,SYS,SYSMAN,WK_TEST,WKSYS,WKPROXY,
WMSYS,XDB,APEX_PUBLIC_USER,DIP,FLOWS_020100,FLOWS_030000,
FLOWS_040100,FLOWS_010600,FLOWS_FILES,MDDATA,ORACLE_OCM,
SPATIAL_CSW_ADMIN_USR,SPATIAL_WFS_ADMIN_USR,XS$NULL,PERFSTAT,
SQLTXPLAIN,DMSYS,TSMSYS,WKSYS,APEX_040000,APEX_040200,
DVSYS,OJVMSYS,GSMADMIN_INTERNAL,APPQOSSYS,DVSYS,DVF,
AUDSYS,APEX_030200,MGMT_VIEW,ODM,ODM_MTR,TRACESRV,MTMSYS,
OWBSYS_AUDIT,WEBSYS,WK_PROXY,OSE$HTTP$ADMIN,
AURORA$JIS$UTILITY$,AURORA$ORB$UNAUTHENTICATED,
DBMS_PRIVILEGE_CAPTURE,CSMIG,MGDSYS,SDE,DBSFWUSER
Depending on your Oracle installation, you may have several other
system roles defined. To append these users to the schema exclusion
list, just set the SYSUSERS configuration directive to a
comma-separated list of system users to exclude. For example:
SYSUSERS INTERNAL,SYSDBA,BI,HR,IX,OE,PM,SH
will add users INTERNAL and SYSDBA to the schema exclusion list.
FORCE_OWNER
By default, the owner of the database objects is the one you're
using to connect to PostgreSQL using the psql command. If you use an
other user (postgres for example), you can force Ora2Pg to set the
object owner to be the one used in the Oracle database by setting
the directive to 1, or to a completely different username by setting
the directive value to that username.
FORCE_SECURITY_INVOKER
Ora2Pg uses the function's security privileges set in Oracle and it
is often defined as SECURITY DEFINER. If you want to override those
security privileges for all functions and use SECURITY DEFINER
instead, enable this directive.
USE_TABLESPACE
When enabled this directive forces ora2pg to export all tables and
indexes using the tablespace name defined in Oracle database. This
works only with tablespaces that are not TEMP, USERS or SYSTEM.
WITH_OID
Activating this directive will force Ora2Pg to add WITH (OIDS) when
creating tables or views as tables. Default is same as PostgreSQL,
disabled.
LOOK_FORWARD_FUNCTION
List of schemas to get functions/procedures meta information that
are used in the current schema export. When replacing calls to
functions with OUT parameters, if a function is declared in an other
package, then the function call rewriting can not be done because
Ora2Pg only knows about functions declared in the current schema. By
setting a comma-separated list of schemas as value of this
directive, Ora2Pg will look forward in these packages for all
functions/procedures/packages declarations before proceeding to
current schema export.
NO_FUNCTION_METADATA
Forces Ora2Pg to not look for function declarations. Note that this
will prevent Ora2Pg from rewriting function replacement calls if
needed. Do not enable it unless looking forward at functions breaks
other exports.
Export type
The export action is performed following a single configuration
directive 'TYPE', some others add more control over what should be
exported.
TYPE
Here are the different values of the TYPE directive, default is
TABLE:
- TABLE: Extract all tables with indexes, primary keys, unique keys,
foreign keys and check constraints.
- VIEW: Extract only views.
- GRANT: Extract roles converted to Pg groups, users and grants on all
objects.
- SEQUENCE: Extract all sequences and their last positions.
- TABLESPACE: Extract storage spaces for tables and indexes (Pg >= v8).
- TRIGGER: Extract triggers defined on actions.
- FUNCTION: Extract functions.
- PROCEDURE: Extract procedures.
- PACKAGE: Extract packages and package bodies.
- INSERT: Extract data as INSERT statements.
- COPY: Extract data as COPY statements.
- PARTITION: Extract range and list Oracle partitions with subpartitions.
- TYPE: Extract user defined Oracle types.
- FDW: Export Oracle tables as foreign tables for Oracle, MySQL and SQL Server FDW.
- MVIEW: Export materialized views.
- QUERY: Try to automatically convert Oracle SQL queries.
- KETTLE: Generate XML ktr template files for use by Kettle.
- DBLINK: Generate Oracle foreign data wrapper server to use as dblink.
- SYNONYM: Export Oracle's synonyms as views on other schema's objects.
- DIRECTORY: Export Oracle's directories as external_file extension objects.
- LOAD: Dispatch a list of queries over multiple PostgreSQL connections.
- TEST: Perform a diff between Oracle and PostgreSQL databases.
- TEST_COUNT: Perform a row count diff between Oracle and PostgreSQL tables.
- TEST_VIEW: Perform a row count diff between Oracle and PostgreSQL views.
- TEST_DATA: Perform data validation check on rows on both sides.
- SEQUENCE_VALUES: Export DDL to set the last values of sequences
Only one type of export can be performed at a time so the TYPE
directive must be unique. If you have more than one only the last
found in the file will be registered.
Some export types cannot or should not be loaded directly into the
PostgreSQL database and still require little manual editing. This is
the case for GRANT, TABLESPACE, TRIGGER, FUNCTION, PROCEDURE, TYPE,
QUERY and PACKAGE export types especially if you have PL/SQL code or
Oracle specific SQL in them.
For TABLESPACE you must ensure that file paths exist on the system
and for SYNONYM you may need to ensure that the object's owners and
schemas correspond to the new PostgreSQL database design.
Note that you can chain multiple exports by giving to the TYPE
directive a comma-separated list of export types, but in this case
you must not use COPY or INSERT with other export types.
Ora2Pg will convert Oracle partitions using table inheritance,
triggers and functions. See documentation at PostgreSQL:
http://www.postgresql.org/docs/current/interactive/ddl-partitioning.
html
The TYPE export allows export of user defined Oracle types. If you
don't use the --plsql command line parameter it simply dumps Oracle
user type as-is else Ora2Pg will try to convert it to PostgreSQL
syntax.
The KETTLE export type requires that the Oracle and PostgreSQL DNS
are defined.
Since Ora2Pg v8.1 there are three new export types:
SHOW_VERSION : display Oracle version
SHOW_SCHEMA : display the list of schemas available in the database.
SHOW_TABLE : display the list of tables available.
SHOW_COLUMN : display the list of tables columns available and the
Ora2PG conversion type from Oracle to PostgreSQL that will be
applied. It will also warn you if there are PostgreSQL reserved
words in Oracle object names.
Here is an example of the SHOW_COLUMN output:
[2] TABLE CURRENT_SCHEMA (1 rows) (Warning: 'CURRENT_SCHEMA' is a reserved word in PostgreSQL)
CONSTRAINT : NUMBER(22) => bigint (Warning: 'CONSTRAINT' is a reserved word in PostgreSQL)
FREEZE : VARCHAR2(25) => varchar(25) (Warning: 'FREEZE' is a reserved word in PostgreSQL)
...
[6] TABLE LOCATIONS (23 rows)
LOCATION_ID : NUMBER(4) => smallint
STREET_ADDRESS : VARCHAR2(40) => varchar(40)
POSTAL_CODE : VARCHAR2(12) => varchar(12)
CITY : VARCHAR2(30) => varchar(30)
STATE_PROVINCE : VARCHAR2(25) => varchar(25)
COUNTRY_ID : CHAR(2) => char(2)
These extraction keywords are used to only display the requested
information and exit. This allows you to quickly explore on what you
are going to work with.
The SHOW_COLUMN allows another ora2pg command line option: '--allow
relname' or '-a relname' to limit the displayed information to the
given table.
The SHOW_ENCODING export type will display the NLS_LANG and
CLIENT_ENCODING values that Ora2Pg will use and the real encoding of
the Oracle database with the corresponding client encoding that
could be used with PostgreSQL.
Ora2Pg allows you to export your Oracle, MySQL or MSSQL table
definitions to be used with the oracle_fdw, mysql_fdw or tds_fdw
foreign data wrapper. By using type FDW your tables will be exported
as follows:
CREATE FOREIGN TABLE oratab (
id integer NOT NULL,
text character varying(30),
floating double precision NOT NULL
) SERVER oradb OPTIONS (table 'ORATAB');
Now you can use the table like a regular PostgreSQL table.
Release 10 adds a new export type designed to evaluate the content
of the database to migrate, in terms of objects and cost to complete
the migration:
SHOW_REPORT : show a detailed report of the Oracle database content.
Here is a sample report: http://ora2pg.darold.net/report.html
There is also a more advanced report with migration cost. See the
dedicated chapter about Migration Cost Evaluation.
ESTIMATE_COST
Activate the migration cost evaluation. Must only be used with
SHOW_REPORT, FUNCTION, PROCEDURE, PACKAGE and QUERY export types.
Default is disabled. You may want to use the --estimate_cost command
line option instead to activate this functionality. Note that
enabling this directive will force PLSQL_PGSQL activation.
COST_UNIT_VALUE
Sets the value in minutes of the migration cost evaluation unit.
Default is five minutes per unit. See --cost_unit_value to change
the unit value at command line.
DUMP_AS_HTML
By default when using SHOW_REPORT the migration report is generated
as simple text. Enabling this directive will force ora2pg to create
a report in HTML format.
See http://ora2pg.darold.net/report.html for a sample report.
DUMP_AS_JSON
By default when using SHOW_REPORT the migration report is generated
as simple text. Enabling this directive will force ora2pg to create
a report in JSON format.
See http://ora2pg.darold.net/report.html for a sample report.
DUMP_AS_CSV
By default when using SHOW_REPORT the migration report is generated
as simple text, enabling this directive will force ora2pg to create
a report in CSV format.
See http://ora2pg.darold.net/report.html for a sample report.
DUMP_AS_FILE_PREFIX
By default when using SHOW_REPORT the migration report is generated
to stout. Enabling this directive in conjunction with DUMP_AS_*
directives will force ora2pg to create a report files with the given
extensions and formats. This option allows you to combine multiple
DUMP_AS_* formats.
See http://ora2pg.darold.net/report.html for a sample report.
HUMAN_DAYS_LIMIT
Use this directive to redefine the number of person-days limit where
the migration assessment level must switch from B to C. Default is
set to 10 person-days.
JOBS
This configuration directive adds multiprocess support to COPY,
FUNCTION and PROCEDURE export types. The value is the number of
processes to use. Default is to disable multiprocessing.
This directive is used to set the number of cores to use to
parallelize data import into PostgreSQL. During FUNCTION or
PROCEDURE export type each function will be translated to plpgsql
using a new process. The performance gain can be very important when
you have tons of functions to convert.
There's no limitation in parallel processing other than the number
of cores and the PostgreSQL I/O performance capabilities.
Doesn't work under Windows Operating System, it is simply disabled.
ORACLE_COPIES
This configuration directive adds multiprocess support to extract
data from Oracle. The value is the number of processes to use to
parallelize the select query. Default is parallel query disabled.
The parallelism is built on splitting the query following the number
of cores given as value to ORACLE_COPIES as follows: