Hey Luciano,
Without knowing exactly what you want, I tried to come up with something that demonstrates a number of options for you. Run this script in DESIGNrev to see the output. I think it will be more clear if you play around with turning on and off the layers. I put temporary and output layers on the same base layer but with different datatypes to help visualize the problem. The script does this:
1. Creates an example layout with two cells instantiated in a topcell.
The two cells have different geometry on layer 3.
2. Creates temporary layers (tmp1 and tmp2) in both cells that covers each cell entirely.
3. Creates a temporary layer (tmp3) that covers the entire top cell.
3. ORs the two cell layers onto a new temporary layer (tmp4) with the hier flag on so it propagates up to the top cell.
4. NOTs the combined tmp4 layer with the tmp3 layer to generate a layer (top_output_outlines) that entirely surrounds the cells.
This green layer might be what you're looking for. If you're also looking to generate a layer that surrounds the geometry inside the cells, I've added a few more operations to the script that do that:
5. Temporary layers in Step 2 are NOT'd with the design layer and OR'd together to get a layer (tmp5) that covers the empty space in each cell.
6. The new layer is NOT'd with the temporary layer (tmp3) that covers the entire top cell. This produces a layer that surrounds the design as if it was flat.
The cellC_output and cellB_output layers are just the NOT of the geometry in each cell.
There are plenty of ways to do this, so I'm sure someone could come up with a different solution.
Here's the script (just copy and paste it into a text file and call calibredrv myScript.tcl):
##############################################################################
# BEGIN
##############################################################################
#create new layout
set ll [layout create]
#====variables=======
#cell names
set topcell TOP_CELL
set cellA cell_A
set cellB cell_B
set design_layer 3
#temp layers
set tmp_layer1 5.1
set tmp_layer2 5.2
set tmp_layer3 5.3
set tmp_layer4 5.4
set tmp_layer5 5.5
#generated overlay layer
set outlayer1 50.1
set outlayer2 50.2
set outlayer3 50.3
set outlayer4 50.4
#====/variables=======
##########################################################
# This part creates a layout with 2 cells for the example
##########################################################
#create new topcell
$ll create cell $topcell
#create layers in layout
$ll create layer $design_layer
$ll create layer $tmp_layer1
$ll create layer $tmp_layer2
$ll create layer $tmp_layer3
$ll create layer $tmp_layer4
$ll create layer $tmp_layer5
$ll create layer $outlayer1
$ll create layer $outlayer2
$ll create layer $outlayer3
$ll create layer $outlayer4
#create child cells
$ll create cell $cellA
$ll create cell $cellB
#create some random shapes for cellA
$ll create polygon $cellA $design_layer 10 10 100 100
$ll create polygon $cellA $design_layer 120 200 200 100
$ll create polygon $cellA $design_layer 140 50 300 10
#create some random shapes for cellB
$ll create polygon $cellB $design_layer 5 50 95 95
$ll create polygon $cellB $design_layer 0 0 100 10
$ll create polygon $cellB $design_layer 0 20 100 30
#Place the cells as an instances in the topcell
set x 0
set y 0
set mirror 0
set angle 0
set mag 1
$ll create ref $topcell $cellA $x $y $mirror $angle $mag
set x 250
set y 250
set mag 2
$ll create ref $topcell $cellB $x $y $mirror $angle $mag
##########################################################
# Here's where the layer merging and NOTing takes place
##########################################################
#get the bounding box of each cell instance and the top cell
set cellCoordsA [$ll bbox $cellA]
set cellCoordsB [$ll bbox $cellB]
set cellCoordsT [$ll bbox $topcell]
#create a new layer the same size as the bounding box over each cell
eval $ll create polygon $cellA $tmp_layer1 $cellCoordsA
eval $ll create polygon $cellB $tmp_layer2 $cellCoordsB
eval $ll create polygon $topcell $tmp_layer3 $cellCoordsT
#Perform a NOT between the random shapes and the blanket temporary layer to get the difference layer
$ll NOT $tmp_layer1 $design_layer $outlayer1
$ll NOT $tmp_layer2 $design_layer $outlayer2
#process top layer (different ways to do this depending on what you want
$ll OR $tmp_layer1 $tmp_layer2 $tmp_layer4 -hier 1
$ll NOT $tmp_layer3 $tmp_layer4 $outlayer3
#create combinedlayer that considers hierachy of CellA and B blockages
$ll OR $outlayer2 $outlayer1 $tmp_layer5 -hier 1
$ll OR $outlayer3 $tmp_layer5 $outlayer4 -hier 1
#rename layers for readability in OASIS
$ll layernames $outlayer4 top_output_combined
$ll layernames $outlayer3 top_output_outlines
$ll layernames $outlayer2 cellB_output
$ll layernames $outlayer1 cellC_output
$ll layernames $design_layer design
$ll layernames $tmp_layer1 tmp1
$ll layernames $tmp_layer2 tmp2
$ll layernames $tmp_layer3 tmp3
$ll layernames $tmp_layer4 tmp4
$ll layernames $tmp_layer5 tmp5
#create the OASIS file
$ll oasisout output.oas