PCI Bus:

PCI is a 2nd generation bus standard that was developed by Intel in 1990. By 1994, PCI became widely used in Pentium PCs. PCI-X which was an enhanced version of PCI was released with faster speeds. After PCI and PCI-X, came the enhanced PCI Express (or PCIE) which was the 3rd generation bus standard, with much faster speeds. This is what we have today in all modern devices. Original PCI and PCI-X are almost non existent.

There were multiple PCI standard as PCI 1.0 (5V signaling), PCI 2.0 (3.3V signaling, PCI-X) and PCI 3.0 (PCIE).The 2nd generation buses were PCI and enhanced version of PCI called PCI-X. The third generation of PCI is what we have today known as PCI Express (or PCIE). We'll look at 3 standards: PCI, PCI-X and PCI Express. like any other bus standard, PCI main objective is to transfer data between devices at highest rate possible.

Spec:

PCI 1.0: PCI bus originally had 33.33MHz clock with synchronous transfers. They had 32 bit wide bus, which gives it a peak transfer rate of 4byte*33.33=133MB/sec. It's 5V signalling, though 3.3V is also supported. It allowed multiple devices to share the same bus, which put a limit on the maximum frequency the bus can support. Clock speed of 33.33MHz (i.e time period=30ns) was only able to support 4-5 slots per bus, as beyond that load would be too high to meet timing. PCI later increased clock speed to 66MHz, which allowed only 1-2 slots per bus.

PCI 2.0: Later PCI-X came with 66MHz and 133MHz clock speeds, with more slots per bus.

PCI connector:

There are PCI connector cards which have contacts on each side of the connector. They have few notches to make sure that they fit only where they are compatible with the voltage supply (5V or 3.3V).

PCI bus transactions:

PCI bus traffic consists of a series of PCI bus transactions. Each transaction consists of an address phase followed by one or more data phases. The direction of the data phases may be from initiator to target (write transaction) or vice versa (read transaction), but all of the data phases must be in the same direction. Either party may pause or halt the data phases at any point.

Each PCI bus transaction

 

a local computer bus that attaches hardware devices in a

PCI Express:

Documents:

This by mindshare is a good one:

https://www.mindshare.com/files/resources/MindShare_Intro_to_PIPE_spec.pdf

 

Genus:

Genus is the latest Synthesis tool from Cadence released somewhere around 2015. Before that RC (RTL Compiler) was the Synthesis tool from Cadence. Genus supposedly has better correlation with PnR and Timing tools from Cadence, as some of the engines that are used in those tools are used over here in Genus.

Legacy Vs CUI mode:

Genus uses CUI mode by default. Below are some diff b/w DC and Genus cmds:

DC Command
Genus Command
Notes
get_attribute get_property Most of DC "attribute" cmds have corresponding "property" cmds in Genus.
define_user_attribute define_attribute Other Cadence tools (Innovus/Tempus) use define_property (as expected), but surprisingly Genus doesn't support this, it has its own different define_attribute instead.
list_attributes list_property Genus. Tempus and Innovus all use list_property
set_user_attribute set_attribute Again, Genus uses "attribute" instead of "property". Here the argument order is different than in DC, objects need to come last in Genus
alias alias alias A B C works for DC but should be alias A "B C" for Genus. only double quote. Single quote doesn't work
report_path_group report_timing -cost_group * path group doesn't have a separate cmd in Genus
set_path_margin set_path_adjust  
get_cells, get_net, get_pins, get_lib_cells get_cells, get_net, get_pins, get_lib_cells same as DC (these are SDC cmds)
remove_designs * delete_obj /designs/* Genus has virtual linux style dir structure
read_verilog read_netlist same as DC
current_design Itop current_Design Itop same as DC
report_timing report_timing same as DC (these are SDC cmds)
set_false_path, set_multicycle_path, set_case_analysis, .. set_false_path, set_multicycle_path, set_case_analysis, .. same as DC (these are SDC cmds)
     
 SDC cmds are partially supported in genus.
Genus help:
On genus shell, type "help cmd_name" or "man cmd_name" and it will show the details of the cmd. Also, partial cmd name as "help *lib*" will show all cmds with lib in the name. Typing first few letters of cmd followed by a tab shows all possible cmds starting with those letters.
-------

Genus flow:

It's same flow as RC.

run Genus: script run.tcl

LEGACY MODE: genus -legacy_ui -f tcl/run.tcl -log /logs/top.log => prompt shows as genus@root:>

CUI MODE: genus -f tcl/run.tcl -log /logs/top.log => prompt shows as legacy_genus :/>

NOTE: Now you can use genus cmds on prompt. To see gui, type gui_show. (See section on "Cadence cmds" for details on CUI vs Legacy).

 

 

Steps:

1. Read Tech libs:

The default search path for libraries, scripts, and HDL files is the directory in which Genus is invoked. To specify non default path for library, set following attributes:

set_db lib_search_path

 

2. Read RTL:

3. elaborate => converts HDL rep into a netlist and performs High Level Opt (HLO). HLO here include Mux optimizations and Spasreness and redundancy opt

4. Read UPF, Timing constraints, DEF => UPF needs to be read only if there are multiple power domains in the design.

5. syn_gen => synthesize to generic gates. Performs data path and other HLO. Also does Mux opt, bit level opt, generic placer and physical buffering

6. syn_map => maps generic gates to tech gates. Does datapath arch selection, DP opt, Bit level opt, tech mapping, multibit opt and Timing/area CRR

7. syn_opt => performs bit level opt for area and timing

8. scan

9. syn_opt / iSpatial / Congestion prediction =>

 

Flow:

Clock cmds:

  • report_clocks => SDC cmd, uniform syntax
  • report_clock_gating => Reports clock-gating (CG) information for the design, and is useful to see all the statistics on where clk gating is not effectively applied.
    • report_clock_gating > clk_gating.rpt => This shows summarized info for # of CG (Genus vs Non-Genus), # of flops (Genus vs NonGenus gated/ungated), MultiBit Flops, etc. It reports # of leaf CG instances, followed by the # of Multi Stage CG (MSCG) instances (i.e where CG are NOT leaf CG anymore). MSCG has 2 options to indicate where should it cout from => from the flop or from the root.
      • -multi_stage_count_from_flop => this option counts from leaf flop, in which case Level 0 is the CG driving the flop, Level 1 is one level higher and so on.
      • -multi_stage_count_from_root => this other option is for counting from the root clk pin, in which case Level 1 is the top level of CG, level 2 is next lower level, until we get to the flop. NOTE: Here levels start from Level 0, while when counting from flop, we started with Level 1.
  • report_clock _tree_structure -out_file clks.rpt => This reports the clk tree hierarchy, which is very useful to see the whole clk tree structure. It shows different LEVELS of clk tree as L1, L2, etc. Start of a new level is determined if there is a clk gater, RTL instantiated gate, etc in design (i.e anything other than buffer and inverter, unless it's RTL instantiated buf/inv). There are many options to fine tune this report.
    • report_clock_tree_structure -show_sinks -clock_trees {clk1 clk2} => By default cmd shows the clk tree for all clks until the sink. These options show clk tree only for specified clks, with sinks included. NOTE: the same flops may appear under multiple clks, since 1 flop may be driven by multiple clks (depending on clk sdc file)
  • report

 

CPF Syntax:

CPF is very similar to UPF in syntax. Since it's not used widely anymore, I'm listing an example below which shows the basic syntax. You can omit this whole section on CPF.

CPF cmd file Example:

Below is an example CPF file. The top level file is called TOP.cpf.

TOP.cpf =>

set_cpf_version 1.1
set_hierarchy_separator "/"
set_design chip_tb/chip_top/u_dig_top_wrapper/u_dig_top

##create top level PD
create_power_domain -name PD_TOP \
     -default => includes everything in PD_TOP that isn't included explicitly in other domains

##create PD for all domains
#for PDMCU ( pdmcu = u_imp in arm_core + others attached to mcu),
create_power_domain -name PD_MCU \
     -base_domains { PD_TOP } \
     -instances { \
           u_dig_top_pdmcu/u_ahb_to_apb u_ahb_peripheral_system u_dft u_sync_2s \
           u_dig_top_pdmcu/u_cm0plus_subsystem/u_cm0pmtbintegration/u_cm0pintegration/u_imp } \     
     -shutoff_condition {!u_pmu_top/dig0_pm_if_ps_on | !u_pmu_top/dig0_pm_if_ps_good} => Anytime these signals are 0, all o/p ports of above instances goto "x", when running RTL sims with this cpf file (power aware rtl sims).

#similarly for other PD (PD_HFADC)
create_power_domain -name PD_HFADC \
     -base_domains { PD_TOP } \
     -instances { u_apb_peripheral_system/u_dig_adc/x_adc_hf } \
     -shutoff_condition {!u_pmu_top/dig1_pm_if.ps_on | !u_pmu_top/dig1_pm_if.ps_good}

#PD for RAM/ROM IP do not have inst name, since they are mapped later with their cpf file
create_power_domain -name VD_RAM_MTB \
     -base_domains { PD_TOP } \
   -shutoff_condition { !(u_dig_top_wrapper/u_ram/PON&u_dig_top_wrap/u_ram/PGOOD) }


#####################
##create power nets
#primary voltage
create_ground_nets -nets { VSS }
create_power_nets  -nets { VDD   } -voltage 1.08

#switched voltages
create_power_nets -nets { VDD_mcu   } -voltage 1.08 -internal
create_power_nets -nets { VDD_hfadc } -voltage 1.08 -internal

update_power_domain -name PD_TOP \
     -primary_power_net VDD \
     -primary_ground_net VSS

update_power_domain -name PD_MCU \
     -primary_power_net VDD_mcu \
     -primary_ground_net VSS

update_power_domain -name PD_HFADC \
     -primary_power_net VDD_hfadc \
     -primary_ground_net VSS

#create connections
#create pins for PDTOP aand connect it appr
create_global_connection -net VDD -pins { VDDNW }  -domain {PD_TOP}
create_global_connection -net VDD -pins { VDD } -domain {PD_TOP}
create_global_connection -net VDD -pins { VDDC } -domain {PD_TOP}
create_global_connection -net VDDAR -pins { VDDAR } -domain {PD_TOP}
create_global_connection -net VDD -pins { VDDPR } -domain {PD_TOP}
create_global_connection -net VNWA -pins { VNWA } -domain {PD_TOP}
create_global_connection -net VDDS -pins { VDDS } -domain {PD_TOP}
create_global_connection -net VPP -pins { VPP }  -domain {PD_TOP}

#similarly do it for PD_MCU and PD_HFADC
create_global_connection -net VDD_mcu -pins { VDDNW } -domain {PD_MCU}
create_global_connection -net VDD_mcu -pins { VDD } -domain {PD_MCU}
create_global_connection -net VDD -pins { VDDC } -domain {PD_MCU}
create_global_connection -net VDDAR -pins { VDDAR } -domain {PD_MCU}
create_global_connection -net VDD_mcu -pins { VDDPR } -domain {PD_MCU}
create_global_connection -net VNWA -pins { VNWA } -domain {PD_MCU}
create_global_connection -net VDDS -pins { VDDS } -domain {PD_MCU}


#create_global_connection -net VPP -pins { VPP } -domain PD_MCU => for efuse, not needed

#connect switch instances for both MCU and HFADC. Note: RTL for these switches don't have power ports, so we create power pins using this and then connect to allow it to have power aware behaviour. Only Pon, Pgood i/p pins, and ack o/p pins are in RTL of switches.
create_global_connection -net VDD_mcu -pins { VDD }     -instances { u_dig_top/u_sw200u_mcu }
create_global_connection -net VDD     -pins { VDDC}     -instances { u_dig_top/u_sw200u_mcu }

#connect RAM IP pins
create_global_connection -net VNWA -pins { VNWA } -instances {digtop/../u_RAM }
create_global_connection -net VPP  -pins { VPP } -instances {digtop/../u_RAM } => and so on for all power pins


##############
#RAMS/ROMS which have inbuilt support for power switch/isolation. They have separate cpf file which specifies power domain, iso rules etc. See below for such an ex:
-----------------
MY_RAM.cpf

set_macro_model MY_RAM => macro model names as something

# create_nominal_condition -name ON -voltage 0.75 -ground_voltage 0
create_nominal_condition -name ON -voltage 1.08 -ground_voltage 0
create_nominal_condition -name OFF -voltage 0 -ground_voltage 0 -state off

# 3 separate power domains with ports on specified power domains
create_power_domain -name PD_VDDPR_VSS -boundary_ports { FDI FCLRZ PGOOD AON \
    PON RETON SO1 SO0 FCLK OFFP AGOOD FDO RETGOOD Q[31:0] } -default => default, so everything within module is in this PD
create_power_domain -name PD_VDDP_VSS -boundary_ports { ATPGM EZ TM WZ DFT CLK \
    TEZ  WRENZ[31:0] } -shutoff_condition { !(PON&PGOOD) } -base_domains { PD_VDDPR_VSS } => This is swiched PD
create_power_domain -name PD_VDDAR_VSS

create_isolation_rule -name ISOLATION -isolation_output low \
                -from PD_VDDP_VSS -secondary_domain PD_VDDPR_VSS \
                -isolation_condition { !(PON&PGOOD) } \
                -pins {  SO1 SO0 Q[31:0] }

update_power_domain -name PD_VDDPR_VSS -primary_power_net VDDPR    -primary_ground_net VSS
update_power_domain -name PD_VDDP_VSS  -primary_power_net VDDP     -primary_ground_net VSS
update_power_domain -name PD_VDDAR_VSS -primary_power_net VDDAR    -primary_ground_net VSS

create_power_mode -name NORMAL -default -domain_conditions { PD_VDDPR_VSS@ON \
    PD_VDDP_VSS@ON PD_VDDAR_VSS@ON}
create_power_mode -name RETENTION -domain_conditions { PD_VDDPR_VSS@ON \
    PD_VDDP_VSS@OFF PD_VDDAR_VSS@ON}
create_power_mode -name POWER_DOWN -domain_conditions { PD_VDDPR_VSS@OFF \
    PD_VDDP_VSS@OFF PD_VDDAR_VSS@OFF }

end_macro_model
------------------
#now we map above cpf file with IP inst below
set_instance u_dig_top_wrapper/u_dig_top/u_dig_top_pdmcu/u_mica_mtbram/u_ram \
   -domain_mapping { {PD_VDDAR_VSS PD_VDDAR} {PD_VDDPR_VSS PD_TOP} {PD_VDDP_VSS VD_RAM_MTB} } \
-port_mapping { {PON u_dig_top_wrapper/u_dig_top/u_dig_top_pdmcu/u_mica_mtbram/u_ram/PON} {PGOOD u_dig_top_wrapper/u_dig_top/u_dig_top_pdmcu/u_mica_mtbram/u_ram/PGOOD} } \ => maps ports on IP to those in PD domain, needed only if port names differ
   -model MY_RAM => name of model in above cpf file


###############
##power switch rules
#PD_MCU
create_power_switch_rule -name ps_rule_1 \
     -domain PD_MCU \
     -external_power_net VDD

update_power_switch_rule -name ps_rule_1 \
     -prefix ps1_  \
   -enable_condition_1     u_pmu_top/dig0_pm_if.ps_on \
   -acknowledge_receiver_1 u_pmu_top/dig0_pm_if.ps_on_ack \ => these provide ack signals for PA RTL sims. Needed so that state m/c can move forward, if it's waiting for ack signal.
   -enable_condition_2     u_pmu_top/dig0_pm_if.ps_good \
   -acknowledge_receiver_2 u_pmu_top/dig0_pm_if.ps_good_ack

#PD_HFADC
create_power_switch_rule -name ps_rule_2 \
     -domain PD_HFADC \
     -external_power_net VDD

update_power_switch_rule -name ps_rule_2 \
     -prefix ps2_  \
   -enable_condition_1     u_pmu_top/dig1_pm_if.ps_on \
   -acknowledge_receiver_1 u_pmu_top/dig1_pm_if.ps_on_ack \
   -enable_condition_2     u_pmu_top/dig1_pm_if.ps_good \
   -acknowledge_receiver_2 u_pmu_top/dig1_pm_if.ps_good_ack

## isolation rules
#PD_MCU for o/p=High
create_isolation_rule -name iso_rule_high_1 \
     -from { PD_MCU } => apply iso high cells to only 2 pins below
         -pins { u_dig_top_pdmcu/u_cm0plus_subsystem/u_cm0pmtbintegration/u_cm0pintegration/u_imp/SLEEPING \
                 u_dig_top_pdmcu/u_cm0plus_subsystem/u_cm0pmtbintegration/u_cm0pintegration/u_imp/SLEEPDEEP \
          }    \
     -isolation_output high \
         -isolation_condition u_pmu_top/dig0_pm_if.iso_enable \ => These iso rules needed for PA RTL sims, so that o/p ports get correct values, instead of "x". NOTE: internal signals of powered dwn blocks will still be x.
         -isolation_target from  \
         -exclude {}

update_isolation_rules -names { iso_rule_high_1 } \
         -location from \
         -within_hierarchy u_dig_top_pdmcu \
         -prefix ISO1_HIGH_

#PD_MCU for all remianing o/p=low
create_isolation_rule -name iso_rule_3 \
         -from { PD_MCU } -to { PD_TOP PD_HFADC VD_RAM } \ => Only apply iso low cells to o/p signals going from PD_MCU to PD_TOP etc, except SLEEPING and SLEEPDEEP o/p signal. All other o/p signals are w/o iso cells.
         -isolation_output low \
         -isolation_condition u_pmu_top/dig0_pm_if.iso_enable \
         -isolation_target from  \
         -exclude { u_dig_top_pdmcu/u_cm0plus_subsystem/u_cm0pmtbintegration/u_cm0pintegration/u_imp/SLEEPING \
                    u_dig_top_pdmcu/u_cm0plus_subsystem/u_cm0pmtbintegration/u_cm0pintegration/u_imp/SLEEPDEEP \
                  }                              

update_isolation_rules -names { iso_rule_3 } \
         -location from \
         -within_hierarchy u_dig_top_pdmcu \
         -prefix ISO3_LOW_

#PD_HFADC
create_isolation_rule -name iso_rule_2 \
     -from { PD_HFADC } \
     -isolation_output low \
     -isolation_condition u_pmu_top/dig1_pm_if.iso_enable \
     -isolation_target from  \
   -exclude {}

update_isolation_rules -names { iso_rule_2 } \
     -location from \
         -within_hierarchy u_apb_peripheral_system/u_dig_adc/x_adc_hf \
     -prefix ISO2_

###########
## state retention rules
create_state_retention_rule -name ret_rule_1 \
     -domain PD_MCU \ => all flops in this domain are treated as retention flops except for flops in "exclude" modules below
     -restore_edge !u_pmu_top/dig0_pm_if.ret_enable \ => only 1 of save/restore needed
         -save_edge u_pmu_top/dig0_pm_if.ret_enable \ => NOTE: save is opposite polarity of restore
         -exclude u_dig_top_pdmcu/u_ahb_peripheral_system/u_flash_top/u_hardip ... \ => flops in these are excluded as retention
     -target_type flop

###########
##nominal condition
create_nominal_condition -name on \
     -voltage { 1.08 } \
     -ground_voltage { 0 }

create_nominal_condition -name off \
     -voltage { 0 } \
     -ground_voltage { 0 }

update_nominal_condition -name on -library_set QC_MAX_1.2V_INDUSTRIAL_PD_TOP_timing => This lib set defined above comprising of .lib for all cells
update_nominal_condition -name off -library_set QC_MAX_1.2V_INDUSTRIAL_PD_TOP_timing

##power modes
create_power_mode -name run \
     -default \
     -domain_conditions {  PD_TOP@on PD_MCU@on PD_HFADC@on }

create_power_mode -name sleep_hfadc_on \
     -domain_conditions {  PD_TOP@on PD_MCU@off PD_HFADC@on }

create_power_mode -name sleep_hfadc_off \
     -domain_conditions {  PD_TOP@on PD_MCU@off PD_HFADC@off }

## assertion controls
create_assertion_control -name ac1 -domains {PD_MCU} -type suspend

## This gets the cell to rerun "initial" block, when it powers up. This allows us to get init values of mem again.
set_sim_control -action power_up_replay \
         -targets { * } \
         -instances { \
            u_dig_top_wrapper/u_dig_top/u_efuse_wrap \
         } \
         -modules {
            ROM_INST \
         }

###############
##define lib sets
define_library_set -name QC_MAX_1.2V -lib {a_W_150.lib ... c.lib}
define_library_set -name QC_MIN_1.2V -lib {a_S_-40.lib ... c.lib}

update_nominal_condition -name on \
     -library_set QC_MAX_1.2V_INDUSTRIAL_timing

#include other tech cpf files
include "/db/.../CORE_RET.cpf" => CORE_RET.cpf has defines for state retention flops
i.e:  (similarly for all other retention flops)
define_state_retention_cell -cells { RET_CELL_X1 } -cell_type \
    CLK_HIGH -clock_pin CLK -save_function ~RETZ -power_switchable VDD -power \
    VDDC -ground VSS

include "./CPF/MY_RAM.cpf" => CPF files for RAM/ROM etc

end_design

 

Running Power Aware Sims in RTL:

To run power aware sims in irun, add these extra args:
irun ....
             -lps_cpf /db/dig_top_rtl_pasim.cpf \ => cpf file
                     -lps_pmode \
                     -lps_isoruleopt_warn \
                     -lps_isofilter_verbose \
                     -lps_logfile lps.log \
                     -lps_pa_model_on \
                     -lps_stime 1us \
                     -lps_verbose 3  \
                     -lps_iso_verbose \
             -lps_verify \ => to turn on asserts in low power mode

Power Intent:

If you recall RTL coding in any HDL language, there is no way to model power connections in RTL. HDL langauges never had power definitions in them. Earlier in 1980's when HDL were being developed, all chips had single power, so there was no need to specify power connections at RTL level. These power connections to std cells were done in physical implementation phase, by the tools. It was very easy to do these, as power lines, VDD and VSS would just connect to power ports of std cells. The power grid for VDD, VSS would finally come to the pads of chip as ppower pins.

Earlier, power was not a concern, so power specific cmds in HDL were never considered. However, now power is a big concern, and a lot of effort goes into reducing power. This involves reducing lkg power of chip, by turning off portions of chip when not in use. Also, multiple voltage supply rails are being used, whose voltage range is dynamic. This asks for putting power definition in RTL to specify what logic is connected to what power supply. However, putting power connections in RTL reduces flexibility of RTL modeling, as it's supposed to focus more on functional aspect. So, standards were developed, which is a separate file with power intent of RTL coded in it, in a syntax called as power formats (PF). This PF file doesn't modify the RTL, but instead adds an abtract layer that defines how various code in RTL are connected to power supply. Simulation tools understand these PF, and simuate design with both RTL and PF in tandem. This is known as "power aware RTL sim" (PARTL Sim). These tools will thrown out an "x" from RTL code, when power supply to that piece of RTL code is OFF.  Thus it allows us to verify our design much before it goes thru physical implememtation. Also, Synthesis/Layout tools can also read this PF file and can know what piece of logic connects to what power supply. That keeps power supply connections easy and automated.

The netlist generated by synthesis/pnr tools can be non PG netlist (netlist which has no power ports for stdcells and modules), or PG netlist (one which has power ports VDD/VSS for stdcells and modules). We can specify what kind of netlist we want generated using options when writing netlist. When we had single power designs, PG netlists were not important as they all connected to single power supply, and didn't provide any extra info. However, now with advent of PF, we more commonly generate PG netlist, which shows power connections to each and every stdcell/macro. This allows us to run power aware simulations on gate level netlist (PAGLS sims). This extra sim allows us to find out if all power connections speciifed in PF file, were indeed done as intended. This sim doesn't need PF file, as all power connections are there in gate level netlist. 

 

Power Standards:

There are 2 PF standards in use today:

1. UPF (Unified Power format): The IEEE 1801 Unified Power Format (UPF) Standard establishes a set of commands used to specify the low-power design intent for electronic systems.

UPF manual can be downloaded from IEEE website. Std started developing in 2009. Current one is IEEE Std 1801-2017

See Design Compiler guide (Page 154) section 6-18 (specifying power intent) for details.

2. CPF: (Common Power format): This Format was introduced by Cadence, but shortly thereafter, UPF was proposed. Since UPF hasd backing of lot more companies, it was chosen as IEEE std, and CPF became less popular. The technical differences between the two formats are relatively minor. CPF is not really needed anymore as all tools support UPF now. I've a section on CPF, but it's for my reference only. You can omit it completely.

 

Terminology used in UPF/CPF:

Using UPF/CPF commands, you can specify the supply network, switches, isolation, retention, and other aspects relevant to power management of a chip design.

Power management (PM):  Power management enables a system to operate correctly in a given functional mode with the minimum power consumption. Implementation of pwr mgmt strategies require PM cells as level shifter, isolation, retention reg and repeaters to ensure that the design works correctly in the presence of diff pwr domains. PM cells may have single set of supply (i.e iso cell in destination PD), dual set of supply (i.e iso cell in source PD, level shifter, retention flops, pwr switch) or even more. Objects supplying pwr can be in various supply states, while objects consuming pwr can be in various power states. Supply state of supply port, supply net or supply set function is a combo of 2 values: state value (which may be OFF, UNDETERMINED, PARTIAL_ON and FULL_ON), and a voltage value (value in uV). Powr states are various legal states that objects can be in. These objects may be supply nets/ports/sets, PD, etc.

Below we briefly define various terms used in PM. We'll cover in more detail in UPF section. The terminology I've below is written for UPF, but applies to CPF as well.

1. Power domain:  It's defined group of elements in the logic hierarchy that share a common set of power supply needs. By default, all logic elements in a power domain use the same primary supply and primary ground, and may share other auxillary supplies such as isolation and retention supplies (explained later). A power domain may be single instance or collection of instances, powered by same supply. We define power domains in UPF to identify separate power regions in RTL. power domain identifies the uppermost inst of the domain. Every instance in design should be part of some power domain.

2. scope and extent: Each power domain has a scope and an extent. The scope is the level of logic hierarchy where the power domain exists. Scope in HDL refers to "region of HDL text where names may be defined".  Usually scope is module, submodule or blocks of statement, which can all be nested. Scope in UPF means the same. The extent is the set of logic elements that belong to the power domain and share the same power supply needs. In other words, the scope is the hierarchical level (instance of a module) where the power domain exists, whereas the extent is what all logic is contained within the power domain. So, with "set_scope" cmd, we specify which module of RTL the UPF file applies to. The scope in UPF is set to top module of RTL. If we want UPF file to be applied to some submodule in RTL, we have to set scope accordingly. Extent allows us to omit/include things within that scope that are prsent or absent in that power domain (i.e may be a set of logic within a submodule is supposed to be connected to some other power supply, in this case extent allows us to omit this from this power domain). Otherwise by default, power domain connects everything that is contained in that design hier to that supply to which that power domain is connected to.

Other way to define scope/extent is this: The instance in the logic hierarchy in which a power domain is defined is called the scope of the power domain. The set of instances that belong to a power domain are said to be the extent of that power domain. This distinction is important: while a given instance can be the scope of multiple power domains (i.e 1 power domain may be nested inside other power domain), it can be in the extent of one and only one power domain (since cell power suplly can only be connected to unique pwr supply, i.e VDD pin of cell can't be connected to 2 pwr supply pins). As a consequence of these definitions, all instances within the extent of a domain are necessarily within the scope of the domain or its descendants

3. nets and ports: Each scope or hierarchical level in the design has supply nets and supply ports. A supply net is a conductor that carries a supply voltage or ground throughout a given power domain. A supply port is a power supply connection point between two adjacent levels of the design hierarchy, between parent and child blocks of the hierarchy. We define supply nets and ports in UPF for power signals, even though these power signal nets/ports are absent in RTL. That is how UPF is able to define power intent of design. For macros or power management cells, these power ports may be specified in liberty or thru 'create_supply_port" upf cmds.

Supply net originates at a root supply driver, which can be an on-chip voltage regulator, an embedded power switch, a bias generator, or an off-chip supply source. One or more of these supply nets connect to a power switch, which has single output supply port. Supply network objects are created independent of power-domain definitions. This allows sharing of common components of the supply distribution network across multiple power domains.

4. Supply set: Related supply nets can be grouped into a supply set, with each supply net in the group providing one or more functions of the supply set. In other words, a supply set represents a collection of supply nets that provide a complete power source for one or more instances. Each supply set defines six standard functions: power, ground, pwell, nwell, deeppwell, and deepnwell. Each function represents a potential supply net connection to a corresponding portion of a transistor. Each function of a given supply set can be associated with a particular supply net that implements the function.

Mostly, power and ground are the only 2 supply nets that are needed (as pwell and nwell connections are tied internally to power/ground). Supply functions of a supply set, and the supply nets they represent, can be connected to instances in one of the following ways: explicitly, automatically, or implicitly. Connections are made downward, from ports or nets in the current scope to ports of descendant instances that are in the extent of the domain.

Supply sets may be primary, retention, isolation or level shifter supply sets, depending on what kind of cell they are meant for.

Supply Set Connections: Supply functions of a supply set, and the supply nets they represent, can be connected to instances in one of the following ways: explicitly, automatically, or implicitly. Connections are made downward, from ports or nets in the current scope to ports of descendant instances that are in the extent of the domain.

  1. explicit connection:
  2. automatic connection: In liberty file, each port has a pg_type as primary power, primary_ground, pwell, nwell, deeppwell, deepnwell. These are connected with matching function names in supply set
  3. Implicit connection: An implicit connection connects the required functions of a supply set to cell instances that do not have explicit supply ports.

5. Supply variation: Supply ports, supply nets, and supply set functions take on values that consist of a state and a voltage. Named port states (see 6.4) and named power states (see 6.5) can be defined to represent the nominal voltages that a supply object may carry. These nominal voltage values are used also for determining whether level-shifting is required (see 6.45). In an implementation, the actual voltage of an object may vary around the nominal values. There are several sources of such variation. One source of variation is the accuracy of the supply. Supply variation can be modeled in UPF using the set_variation command (see 6.53). Supply variation is applied to nominal voltages to derive variation ranges for those voltages. Supply variation ranges (see 4.5.7) are used when determining whether level-shifting is needed.

6. Power states: As explained above, power states represent various operational modes of the domain's supply set. As an ex, a power domain may have 3 states: RUNNING, SLEEP and SHUTDOWN. In RUNNING, all supply sets may be at optimal voltage, while in SLEEPING, they might be at reduced voltages. In SHUTDOWN, primary supply may be OFF, while retention and isolation supplies may be ON). A value of 0 or L in HDL is considered ON or FULL_ON, while a value of 1 or H in HDL is considered OFF. X or Z may be considered UNDETERMINED or PARTIAL_ON depending on settings.

7. power switch: A power switch (or simply switch) is a device that turns on and turns off power for a supply net. A switch has an input supply net, an output supply net that can be switched on or off, and at least one input signal to control switching.

Supply network objects (supply ports, supply nets, and switches) are created within the logic hierarchy to provide connection points for a root supply and to propagate the value of a root supply throughout a portion of the design.


8. level shifter: Where a logic signal leaves one power domain and enters another at a substantially different supply voltage, a level-shifter cell must be present to convert the signal from the voltage swing of the first domain to that of the second domain.

9. isolation cell (aka clamp cell): Where a logic signal leaves a power domain and enters a different power domain, an isolation cell must be present to generate a known logic value during shutdown. If the voltage levels of the two domains are substantially different, the interface cell must perform both level shifting when the domain is powered up and isolation when the domain is powered down. A cell that can perform both functions is called an enable level shifter. Tools can optimize away redundant insertion of iso cells (i.e when nets are tied to constant logic, etc)

Isolation may be inserted for an input or for an output of the power domain. It's placed at i/p so that values coming into the power domain are deterministic even when the driver is powered down. They are needed at o/p so that values coming out of this power domain and going into other domain are deterministic even when driver is powered down. If we think about it, we may infer that isolation cells may need to be provided at i/p ports only, since o/p of a port may fan out to different with different clamp value requirements, so providing iso cells at all i/p ports only may work out. However, if we don't provide iso cells at o/p, then the "unknown" values will be going thru nets to all other blocks. These nets will need repeaters to carry the signal, and unknown inputs may cause short circuit current thru repeaters. So, we provide iso cells at both i/p and o/p of all blocks. Isolation cell's power supply for i/p ports is from same PD supply which is consuming it (as isolation is relevant only when the supply for that PD is on). However, this may be an issue for o/p ports, as they will become floating if driven by same power supply. So, we prefer to drive o/p ports with always on power supply, so that we are guaranteed valid iso values all the time??

An isolation cell operates in two modes: normal mode, in which it acts like a buffer, and isolation mode, in which it clamps its output to a defined value. An isolation enable signal determines the operational mode of an isolation cell at any given time.

9. retention reg/flops and memories: In a power domain that has power switching, any registers that are to retain data during shutdown must be implemented as retention registers. A retention register has a separate, always-on supply net, sometimes called the backup supply, which keeps the data stable in while the primary supply of the domain is shut down. Retention capability can be implemented for both registers and memories.

There are 2 flavors of retention reg:

  1. Balloon-style retention: In a balloon-style retention register, the retained value is held in an additional latch, often called the balloon latch. In this case, the balloon element is not in the functional data-path of the register.Extra save/restore ports are needed to save or restore the values.
  2. Master/slave-alive retention: In a master/slave-alive retention register, the retained value is held in the master or slave latch. In this case, the retention element is in the functional data-path of the register. This style of registers does not have save/restore ports, but may specify a retention condition that could take the register in and out of retention (i.e when power supply turned on/off).

10. Repeaters: Required when distance b/w driver and receiver is large. These repeaters should be on appr power domain.

 

 

Investing in Stocks

As I've indicated in other sections, stocks are the only game in town. Even though Stocks are a ponzi scheme, as we can see based on returns they provide over a 30 year period, they are supported by the full backing and faith of the FED, which is a branch of the government. This branch of government can print infinite money to support the stock market, so stock market will be the winner of all the assets out there.

Should you invest in individual stocks?

There are several reasons why investing in individual stocks should be avoided at all costs. It's a scheme that's followed by a lot of people all over the world, who claim to have made big money, but it's mathematically very very low probabaility event. Of course some people like Warren Buffet have become super rich by investing in individual stocks. We'll discuss that later.

Below are some the reasons for not investing in individual stocks:

1. Predicting future of a company: No one can predict the future of a company, not even the founder of a company. As an example, check Steve Jobs. After introducing ipods in 2003, he kept on exchanging his higher priced stock options for a lower price, and accepting lower number of stock options in the process. He probably believed that his higher priced stock options would be worthless (as the stock may never get to that price). If he knew that his product and the company are going to be successful, he would have probably kept these higher priced stock options, and gotten way more richer. The stocks have gone up by 1000X from his exercise price in 2003. When the founder of the company who has all the insights, doesn't know the future of the company, who are we to predict the success of the company?

Here's another article about Tesla Founder Elon Musk, who on May 1, 2020 claimed that Tesla stock was over valued (even though he was the CEO at that time). If you were smart, you would have sold your Tesla stock, since there's no one who knows Tesla better than Elon Musk. Guess what? The stock went up 8 times from that price in less than 18 months.

https://www.msn.com/en-us/money/news/if-you-invested-1-000-when-elon-musk-said-the-price-of-tesla-shares-was-too-high-here-s-how-much-you-d-have-now/ar-AAWQACD

So this shows that all those stock pickers, wall street firms, etc predicting future of a company are just wasting everyone's time. Some of them will be right some of the time, just due to nature of probablity and statistics, but noone will be right all the time. God and only God knows the future !!

2. Life cycle of a company: There are more than 5K companies in US stock exchange. How do you figure which company you should invest in? Over long term, all companies either get bought out or go bankrupt. Starting from 1955, only 60 of the Fortune 500 companies remained in the list as of 2017. That means most companies got out of business, no matter how successful they were at a time. So, you not only have to invest in the right company, know the future of the company, but also time it so that you can exit at the top price for that company. That's impossible? If you could predict all these, you are already a GOD, and there's no need for money !!

3. Ponzi scheme: If you believe that Stock market is a ponzi scheme, then you should know that the market provides 10%-20% return with FED's support to this ponzi scheme. What FED guarantees is that it will take the market higher by a certain % every year. It doesn't guarantee any individual stock's return. You can think of it as FED saying that they will take the average price of all these stocks (that comprises the stock market) higher by 10% per year. Obviously since the mean is going up by 10%, some stocks will go up by >10%, while some stocks will go up by <10%. It's impossible to know which stocks will go up by more than 10% consistently. FED will save the market, but not any individual company (as it'll be seen as preferred treatment, and potentially illegal activity). Read the article on Fed's "stock market: Is it a Ponzi scheme" and "A dual class society". 

4. There's nothing called Value: You may think that value stocks will be the ones that will give you >10% (as Warren Buffet famously preaches). But these value stocks don't really have any value. Everything is way overpriced with respect to what any sane investor would pay for any of these businesses. The fact that they are trading on a stock exchange by it's very nature implies there is no value. They are all stocks with some random number written on them that determines their price. People start betting on both sides of that number, some betting that the number will go up, while some betting that the number will go down. That determines the daily price. It can keep trading like this till infinity, and the number has no meaning. The only thing that is guaranteed is that the average number of all the stocks will go up by 10% every year. The only real return that you get from any stock is the amount of money that it pays in dividend over your lifetime holding it.

5. No recovery from loss: Let's say you bought an individual stock that eventually went bankrupt or goes down by 90% or so. There is no way that you can recover this loss. Why? Because there's very little money left to invest in anything else. You will need a 900% gain to recover from a 90% loss. That kind of gains can't come in your entire lifetime with any sane investment. The money is almost all gone. No one can declare that a company will never ever go bankrupt. Warren Buffet keeps saying that he picks companies that are going to live for ever, and I've always scratched my head as to how can he guarantee that. To declare perpetual existence of a company, you have to know the future competition, future customers, future CEOs of that company, future shifts in taste of consumers, and a heck lot of future things both inside that company, as well as outside that company. If you can really predict all that, you are a GOD in human form and you don't need to be trading on stock market. You are way beyond that lowly humanly thing !!

Investing in Index Funds?

The only time when you lose money on a stock is when the company goes bankrupt. At that time, the number on that stock (i.e it's price) goes to 0 since it's not going to be traded anymore on the stock market. So, the price just disappears. Most of the companies eventually go bankrupt, so you have a high chance of losing all the money invested in that stock.

If you buy any large basket of stocks, you are piggybacking on the FED. You are basically saying that you will take the guaranteed return that FED is providing you with 0 risk. There's no science or IQ involved in choosing stocks. They are just random papers with some numbers written on them, that determine their price. The sum of all of these numbers is guaranteed by FED for a yearly +ve return, so stick to that.

This is how to think of the market return. Plot a graph, with Y axis being the % return per year and X axis being the year. FED puts a horizontal line with a Y=10%, implying stocks increase by 10% for each year. Now on any given day, there will be bunch of stocks with returns higher than 10%, and many with returns lower than 10%.

 FIXME: Put a graph ?

 


 

Stock trading in USA:

Once you have brokerage account opened (see details in brokerage section), you are ready to start trading. Before we get into details, let's see which are biggest holders of stocks. Ultimately it's we people who own most of the stocks in the world, but we usually buy it one of the 3 forms: as Mutual Fund (MF), ETF or individual stocks. Mutual Fund and ETF are explained below. These Mutual Fund and ETF are issued by some company and they are the ones who are listed as having owneership stake in these companies. These companies become the biggest owners of many of these public companies, and are listed as majority holders, even though in reality, their ETF/MF are ultimately owned by people.

The US stock market is $40T as of 2023. These are the 10 Companies with largest Assets under Management (AUM):

10 Largest Asset Management Companies in the World - Largest.org

 

Stocks to buy:

Well, as we learned in above section, there is no rationality in buying any individual stock. You should buy the whole market, or any index which represents a significant portion of the market. W500, SPY500, DJIA or anything like this will work. You can go for a limited index as index of 30 stocks, or an index that captures atleast 25% of the total market cap. So, top 10 stocks of the US market will also be fine as an index, if you want to do it on your own (i.e buy these top 10 stocks in proportion to their market cap, and adjust them every few days, but it's just lot of work). So, we'll buy ETF or mutual funds that charge a small fee, but allow us to trade a basket of stocks as 1 stock.

ETF vs Mutual Fund (MF):

A lot of money goes into buying stocks via mutual funds and ETF. As of 2020, total net assets of mutual funds registered in USA was $24T in 2020, compared to $5.5T in 1998. In contarst total net asset of ETF is $5.5T in 2020, compared to $0.1T in 2002. Of the total ETF market, 80% is equity ETF, 15% is Bond ETF, and remaining 5% is misc as commodities, convertibles, etc. Just in 2020 alone, $0.5T of money went in equity ETF, and as of Dec 2021, $0.8T of new money has already made it's way into equity ETF, which are both a record. So, you can see that ETF are growing in popularity much faster than mutual funds. A big reason for that is that ETFs can be traded anytime of day, which allows daytraders to get in and out with pretty low expenses.

ETF and Mutual Funds are not very different. They have a basket of stocks. They put this basket under a ticker symbol. You buy or sell this basket under the ticker symbol of that Fund. It's just like trading an individual stock. However, individual stocks don't charge you any annual fees. But these funds charge you an annual fee (known as expense ratio) which is a certain % of the fund amount. ETF charge you much lower fees (<0.1%) than Mutual Fund (usually 0.5%). Also, ETF can be bought or sold on the trade exchange anytime during the day, and their prices change continually. However, mutual funds only trade after market close at the set price. So, it's advantageous to buy ETF, as you can buy/sell at certain price.

Active Vs Passive:

Both ETF and MF can be actively managed by a group of professionals or be passively managed by a computer.

Actively managed ETFs and MF charge higher expense ratio as they need to pay a bunch of people managing ETF/MF. These people decide what all stocks to keep in the ETF/MF and keep on readjusting it.

Passively managed ETFs and MF charge lower expense ratio as they not managed actively by a person, but rather by a computer. They are also called index funds or index ETF as they track a particular index like S&P500, DOW30, or something like that. These ETFs and MF just mimmick those index, but have to pay some licensing fee to the company holding the rights to that index.

Most of the actively managed ETF and mutual funds are not able to beat an index fund, so no reason to pay a bunch of people managing ETF or mutual fund. They are wasting their time and your money.

Conclusion: Buy passively managed ETF and MF tracking a wide market index (i.e S&P500) as they have lower expense ratio. 90% of actively managed MF/ETF are not able to beat passive index funds. Among MF and ET, go for ETF as they not only charge lower fees, but also give you the control over buying/selling price.

There are thousands of ETF floated by various companies, mimicking various indices.These are the largest ETF companies, issuing a lot of ETF:

https://www.investopedia.com/articles/investing/080415/5-biggest-etf-companies.asp

  • Blackrock (issuer of iShare ETF), Vanguard (largest ETF are VTI and VOO) and State Street (issuer of SPY ETF) are 3 biggest ETF issuers in US capturing 70% of US ETF market.
  • Invesco (issuer of QQQ ETF) is at number 4. Charles Schwab comes at number 5, but is rapidly capturing market, with it's lowest "expense ratio" ETF. "SCHD" (US Dividend Equity ETF) is the largest ETF issued by Schwab.

Vanguard and Charles Schwab provide lowest cost ETF in general. When looking for any ETF, you will always find lowest fee ETF in that category to be from Vanguard or Schwab, so easy decision there. Vanguard has been known to be a leader in low cost ETF and MF, but schwab and others have been catching up lately.

This is the website with full details of all ETF: https://etfdb.com/etfs

This is a list of largest ETF: https://etfdb.com/compare/market-cap/

  • Total ETF AUM globally was $10T as of May, 2023. These include all kinds of ETF as Equity, Bond, etc. In terms of asset class focus, equity ETFs currently compose 76% of the global ETF AUM, fixed income 21%, commodities ~3%, and currency and multi asset ETFs ~0.5%. Total Stock market cap is about $100T. So Stock ETFs have about 7% of the total equity market.
  • US ETF AUM were $7T as of May, 2023 with ~3100 ETF listed on US exchanges. European market has $1.6T, while Asia region has $1.2T in ETF AUM. So, 70% of global ETF are actually US based.
  • Top 3 ETF are all passive index tracking S&P500 => SPY, IVV and VOO totaling $1T in AUM. That speaks about the importance of S&P500 as an index compared to any other index of the world.
  • Most traded ETF are TQQQ, SPY, QQQ with daily volume being > 10% of their ETF count.

These are terms to know when trading ETF:

  • Expense ratio: Different ETF charge different expense ratio. Usually you should go with one with the lowest expense ratio. Expense ratio of most ETF is much lower those of equivalent MF. Some large Fidelity and Vanguard MF actually have expense ratio of 0.01%-0.02%, which is lower than equivalent ETF, so there are exceptions here and there. The only risk with "Expense ratio" is that it may change from time to time, so we are at the mercy of ETF issuers. Historically, Vanguard and Schwab have never raised their "Expense ratio" for their top ETF, so hopefully, we'll never have to liquidate those and move somewhere else.
  • Spread: Other aspect of ETF is liquidity. If a ETF is highly traded, it's spread b/w buy and sell will be very small, so you don't lose much money when buying/selling these. However, if it's thinly traded (because not too many people have this), it may have a larger spread of say 10 cents/share or so. Let's say you buy a ETF at $500/share which has expense ratio of 0.1%. So, you pay a yearly fees of 50 cents/share. Now, if every time you buy or sell and you have a spread of 10 cents, your yearly cost is suddenly 70 cents/share (assuming you buy/sell once every year). This spread will eat into your profits.Also, a lower priced ETF will have larger spread as a percentage of ETF price than one with a higher price. So, it's advantageous to buy ETF with larger price. However, if you are a log term term buy and hold investor, then spreads don't matter as it's only 1 time expense. Expense ratio matter more, so you should go with low expense ratio ETF at cost of higher spread.
  • Premium/discount: ETF have NAV value which is the value of all stocks in that ETF. Most of the times ETF will trade at exactly the same price as it's underlying stock holdings. But sometimes there is a small discrepancy. If the ETF trades for less than it's underlying NAV, it's said to be trading at a discount, while for the other way around, it's said to be trading at a premium. You should buy ETF on a down day, as you may get some discount. Here's more detail: https://www.fidelity.com/learning-center/investment-products/etf/premiums-discounts-etfs
  • Dividend: The way ETF pay dividend is at the end of the quarter on a certain date. However, individual stocks underlying the ETF pay dividends thru out the quarter at different times. By not paying the dividend as soon it's avilable, we lose that opportunity to reinvest that dividend earlier. This dividend meanwhile sits in the ETF trust earning 0% interest. Dividend history of any ETF or stock can be found at: https://www.dividend.com

 


 

Best Equity Mutual Fund:

Even though I don't recommend Mutual funds in general, you will be surprised to know that there are a few "no expense" MF. There are also few MF whose expense ratio is actually lower than equivalent ETF, so it may be beneficial to own these MF over similar ETF. The only downside is that you can only buy it at closing price.

I've included few low cost MF below.

1. Fidelity "zero expense" Index funds:

In 2019, mutual fund giant Fidelity Investments shocked the market when it launched the first mutual funds with a zero expense ratio. Fideltiy introduced 4 "zero expense" index mutual funds.

  • Fidelity ZERO Large Cap Index Fund (MUTF:FNILX) : FNILX “seeks to provide investment results that correspond to the total return of a broad range of large-capitalization U.S. companies" as per Fidelity. It has 506 stocks and looks similar to S&P500
  • Fidelity ZERO Total Market Index Fund (MUTF:FZROX): FZROX "“seeks to provide investment results that correspond to the total return of a broad range of publicly traded companies in the US,” as per Fidelity. It has 2500 stocks and looks similar to Wilshire 5000.
  • Fidelity ZERO International Index Fund (MUTF:FZILX: FZILX is an international counterpart of the domestic FNILX. It is a blend of developed and emerging markets with the latter representing over 20% of the fund’s weight. It has a 38% weight to Europe, while Japan and the U.K. combine for over 28%.
  • Fidelity ZERO Extended Market Index Fund (MUTF:FZIPX): FZIPX focuses on mid- and small-cap stocks. It has 2000 stocks, and basically excludes the S&P 500 from its roster.

2. Fidelity 500 index fund (FXAIX):

This index fund tracks S&P500. Even though this is an index mutual fund, it's expense ratio is pretty small at 0.01%. It's even lower than the lowest cost ETF. It's provided by Fidelity.

3. Vanguard Institutional 500 Index Trust (VFFSX):

This is another mutual fund tracking S&P500 similar to Fidelity's FXAIX. It's expense ratio is also small at 0.01%.

4. Vanguard Total Stock Market Index Fund Institutional Plus Shares (VSMPX):

This is yet another MF tracking total stock market, whose expense ratio is small at 0.02%.

 


 

Best Equity ETF:

You should look to buy ETF with expense ratio < 0.1%. On $1M in stocks, 0.1% expense ratio would cost you about $1K/year, which is not great but still OK. There are few "zero cost" ETF too, though not that widespread.

"Zero" expense ETF:

This article tries to compare "zero expense" ETF with similar "low cost" ETF. Verdict is that the spread in these "zero expense" ETF more than eats up the "free part" and you end up paying more. Many of these 0% expense ratio funds are gimmick, since they raise the fees later. Or they are introductory, where they are "zero" for a while, but go to regular fees after some time.

https://etfmathguy.com/zero-and-negative-expense-ratio-etfs-have-arrived

As of Apr 2022, six ETFs charge no expense ratio, according to ETF Database. One of them is Bond ETF (BKLG), while 2 of them are Exchage Trade Notes (ETN) and NOT ETF. ETN don’t hold the securities in an index, but are unsecured debt securities that promise to pay the return of an underlying index minus the expense ratio. The Pacer iPath Gold ETN (GBUG), which seeks to track the return of the Barclays Gold 3-Month Index Total-Return; and the iPath Silver ETN (SBUG), which tracks the Barclays Silver 3 Month Index Total Return Index.If we leave these 3 aside, below are the 3equity ETF with "zero fees".

  • BNY Mellon U.S. Large Cap Core Equity ETF (BKLC): This is the most widely avialable ETF with no expense ratio, i.e expense is 0%. However, the price of this ETF is about 1/6th the price of similar SPY ETF. This results in a larger spread. As an ex, consider SPY ETF at $500 and BKLC at $80. A spread of 5 cents in buy/sell bid results in 5X worse loss in BKLC. So, what you save in expense ratio, is lost in the spread when buying this. Also, the dividend in this is lower than the SPY ETF. So, I wouldn't recommend this ETF, even with it's 0% expense ratio. You need to get at least the same dividend as S&P 500, or else it's not worth the risk.

  • Next are the 2 SoFi ETF. Both have temporarily waived fees until at least June 30, 2020. So, these are dubious, as they will raise fees, and we won't know by how much. So, best to avoid these too.
    • SoFi Select 500 ETF (SFY): Similar to the S&P 500 Growth Index in that it is composed of the 500 largest publicly traded U.S. companies. Similar to the iShares S&P 500 Growth ETF (IVW)
    • SoFi Next 500 ETF (SFYX): Composed of 500 mid-cap U.S. companies weighted on not just market capitalization but three growth signals. This is a similar fund to the SPDR S&P MIDCAP 400 ETF (MDY)

Below are best low cost equity ETF to buy (The lowest cost ETF are around 0.03%):

ETF mimicking S&P 500 index:

These mimick S&P 500 index. Their expense ratio can't go below 0.03% as this is the licensing fees ETF have to pay to S&P. Their yield is about 1.3% as of 2021.

1. SPDR S&P 500 index ETF (SPY):

This is the most popular ETF, but it has a higher expense ratio of 0.09%. You can can get same S&P500 ETF with less than half the fees, so don't buy this. In past there weren't any lower cost alternative, so this ETF thrived, but now you have cheaper options, with same return. However, it's stil the largest equity ETF as of 2021, because most people hear about this ETF only and don't know that alternatives exist.

2. ishares core S&P 500 ETF (IVV):

This is similar to SPY, but with a lower expense ratio of 0.03%. This is one of the lowest cost ETF mimicking S&P 500, so go with this, or with the next one.

3. Vanguard S&P 500 ETF (VOO):

This is similar to IVV, and it's expense ratio is also 0.03%. I assume IVV may be better, as the price of IVV is 10% higher than that of VOO, so you lose a little less in the spread. Personally, I've bought VOO, as the difference is probably negligible. Also, VOO is product of Vanguard, which is what I personally prefer.

 

ETF mimicking Nasdaq 100 index:

As we saw in our earlier section that Nasdaq has beaten S&P500 over all periods of > 10 years of holding, It lost to S&P 500 only when you bought at peak of stock market in 2000. So, if the stock market (i.e Wilshire 5000) is more than 10% down from it's peak, then it's time to start putting some money in Nasdaq ETF, as you are guaranteed higher returns than S&P500 (assuming FED will pumping money into this ponzi scheme). Nasdaq ETF in general have much higher expense ratio (I'm not sure why, since it's also passively managed). However dividend yield for this is only 0.5% as most stocks don't pay any dividend. Below are ETF mimicking nasdaq 100 (which is a subset of Nasdaq stock market).

1. Invesco QQQ Trust ETF (QQQ):

This is the most popular ETF for Nasdaq 100, but it has a higher expense ratio of 0.2%. It's structured as a trust fund. I don't know of any difference it makes for an investor. This is among the top 5 equity ETF. It has high liquidity. It's spread is small at 1 cent. It used to trade under symbol "QQQQ" but since has dropped 1 Q.

2. Invesco Nasdaq 100 ETF (QQQM):

This is issued by same company and is same as QQQ except that it's expense ratio is 0.15%, and it's an open ended ETF. It is called as "Triple Q Mini". It has a larger spread of 3 cents as it's thinly traded (it has < 1% of AUM than QQQ) . However, QQQM is preferred and there is no reason to hold QQQ in favor of QQQM. QQQM is ideal for long term investors, and gradually adding to this ETF at drops of 10% or more of the market (i.e 10% drop in S&P500) would serve well in stock market ponzi scheme.

 

ETF mimicking Total US Stock market index:

We may also look into US total equity market index mimicking Wilshire 5000.

1.Vanguard Total Stock Market ETF (VTI):

Expense ratio is 0.03%, and this is among the top 5 equity ETF (as of Jan, 2022, expense ratio shows as 0.02% on most websites, probably, because the index has gone up, while their annual costs might have remained the same). However, the yield here is 10% lower than S&P500, and the difference in return b/w this and S&P500 is small. So, better to stick with S&P500 over this. There is an excellent article comparing it to SPY:

https://seekingalpha.com/article/4438495-spy-vs-vti-etf-better-buy

2. Schwab US Broad Market ETF (SCHB):

Similar to VTI, but tracks only 1500 companies. Expense ratio is 0.03%, and it has Schwab's name behind it.

 

Other low cost or high return ETF:

1. Ark Innovation ETF (ARKK):

Ark Innovation has couple of ETF, but this is the one that is the most popular and hot ETF. It's expense ratio is high at 0.75%. It was incepted in 2014, and in last 8 years, it has given returns that blow even Warren Buffet's returns. It's rock star manager, Cathie Woods actively manages this ETF, and tries to find highly innovative and disruptive companies anywhere in the world that will yield insane returns. I'm not a fan of this ETF, as this ETF was formed during an equally insane bull market, so basket of risky junk stocks will almost always will give you better returns than sane ones. This ETF needs to go thru 30%-50% market correction in order to find out it's real returns.

UPDATE: As of Jan 2022, ARKK is already down 55%, while S&P is down only 5% from it's peak. That speaks to the insane returns that a bull market can provide to non-sense funds. If it wasn't for Tesla stock in it's holdings, ARKK would have given 8 year return less than that of S&P500. Most of these ETFs get lucky in 1 stock that gives them 100X or 1000X returns. In such cases, even if all the other stocks go bankrupt, the fund is still able to deliver a decent return, and their manager get a rock star status and a multi million dollar pay check.

2. RoundHill's Magnificent 7 ETF (MAGS):

This is the Mag 7 ETF (7 stocks being AAPL, MSFT, NVDA, TSLA, META, GOOG, AMZN). It provides equal weighting to all stocks (though tsla is at 10%, while others are at 15%). Launched in 2023, it's expense ratio is 0.29%. Not worth it, as it's not diversified enough. It came into existence when biggest US companies were giving the biggest gains in stock market. Over time, it will give almost same return as Nasdaq 100.

3. iShare Top 20 US Stock ETF (TOPT):

This is exactly what the name says => Top 20 US stocks by market cap. This was launched in Oct, 2024, on same lines as Mag7. However, this ETF is diversified enough as top 20 US companies make up ~ 50% of S&P500 market cap.  Expense ratio is 0.2%, so similar to QQQ, but not as diversified. Link => https://www.ishares.com/us/products/339779/ishares-top-20-u-s-stocks-etf

4. Schwab US Dividend ETF (SCHD):

This ETF, introduced in 2011, tracks Dow Jones US Dividend 100 index, which includes 100 U.S. stocks that exhibit consistent dividend payments. It's universe only includes firms with 10+ yrs of dividend history, along with strong fundamentals, low payout ratio, sustainable dividends, etc. Due to this approach, SCHD very easily beats SPYD. It's market cap weighted, and dividend yield is > 4% as of Apr, 2025. It's expense ratio is lower at 0.06%. If you are looking for dividends, this is the best dividend ETF to own, as dividends have increased by > 11%/yr. However SCHD has underperformed VOO over the last 10 years. However, if you are looking to get 4% yield because you can't live on 1% yield on S&P500, then instead of bonds, you should buy this ETF.

SCHD Vs VOO: https://portfolioslab.com/tools/stock-comparison/SCHD/VOO

Link on SCHD pros => https://seekingalpha.com/article/4773173-schd-buy-the-dip-and-call-it-a-day

5. State Street S&P500 High Dividend ETF (SPYD):

This ETF, introduced in 2015, takes the 80 highest yielding Stocks in S&P500. The only difference compared to S&P500 is that it applies equal weighting to all 80 components, so all 80 stocks have weighing of 1%-1.5%. Expense ratio is low at 0.07%. Dividend is high at 4.5% as of 2025. This ETF has gone up by 50% in last 10 years, which looks pretty low, but once you add back reinvested dividends, the yield is ~10% from 2015-2025, which is still lower than SPY return of 12%. .However, it's much less volatile, due to it's high dividend. This ETF also has REITs in it, so it's price appreciation is very low. No added value in this ETF. This ETF is highly risky as it's yield varies considerably. Better dividend paying alternative is SCHD ETF above. 

6. Vanguard High Dividend Yield ETF (VYM):

This ETF introduced in 2006, tracks FTSE High Dividend Yield Index. It has > 500 stocks, so is more diversified, However, yield is considerably lower than SCHD, and total return has also been lower compared to SCHD. Expense is same as SCHD, so no reason to buy VYM over SCHD.

 


 

Leveraged ETF:

Leveraged ETF are latest rage (as in leve-rage), as they amplify your returns by a factor > 1. So, if you lose 10% in a day, a 3X levereged ETF will lose 30% for the day. Conversely if you gain 10% in a day, a 3X levereged ETF will gain 30% for the day. You have all kinds of leveraged ETF from 2X to 4X with all popular indices in both long and short direction. Last 15 years (since 2009) have seen a big bull market with no corrections (except for 2 corrections lasting for < 6 months each), which have given compounded returns of >30% for these "long" ETF. These, as the name implies, are long ETF, i.e they go long on the stocks, meaning your returns are in same direction as the un leveraged ETF. There is also short ETF, which goes in opposite direction of the underlying non-leveraged ETF, i.e if the underlying ETF goes up, your leveraged goes down and vice versa. This is essentially like shorting the market, except that you are leveraging to amplify the return. Since there is no free lunch, and someone else is taking the risk to amplify your returns, these ETF charge you a lot higher fees, usually at 1%.

The below article shows how much return an ETF named TQQQ which is a 3X leveraged version of QQQ ETF has given to it's holders.

https://www.optimizedportfolio.com/tqqq/

As can be seen it gave 50% annual return to it's holders since it's inception in 2011. A $10K sum invested in TQQQ turned into $600K in 10 years.However if you had invested your money in the TQQQ ETF right before 2000 crash, you would not recover your money in your lifetime. However, as shown in link above, if you had hypothetically invested in TQQQ in 1980, you would have managed to get 4X the return of Nasdaq, even with all the ups and downs. So, why are leveraged ETF not considered long term investement?

Reason is that leveraged ETF will eventually go down down to 0, over long time horizon, so you are essentially guaranteed 100% loss. Why it happens, is loss due to ups and downs of market, and these ETF being adjusted on a daily basis. You can think of ups and downs as oscillations, which take energy out of these leveraged ETF. Market going up increases the energy in leveraged ETF, but these oscillations keep on sucking energy out. Larger the oscillations, larger the energy sucked. Eventually, oscillation energy may overwhelm the energy due to markets moving higher. More the ups and downs, quicker will the leveraged ETF go down to 0.

Since 2009, rules of leveraging have changed. FED has now guaranteed that markets will only go higher, the oscillations will be small, and they will move quick to recover any losses happening in market. In fact, FED is encouraging leveraging by providing money at close to 0% interest. This means that leveraged ETF may keep on outperforming non leveraged ETF as long as the FED has the power to print money. However, if a sane government comes into rule, there may be no FED, nor any of their money printing press. So, we should be cautious with these extremely dangerous products. They have the risk to wipe you off completely if sanity ever returns.

This article discusses some more: https://seekingalpha.com/article/4378063-tqqq-is-high-risk-low-reward-investment

So, the lesson with leveraged ETF is to use them for only a very small portion of your portfolio, and that also after a 30% crash from the top.

These are few leveraged ETF: The link shows the top ones: https://etfdb.com/etfs/leveraged/equity/

ProShares Ultra are 2X leveraged, while ProShares UltraPro are 3X leveraged. If they don't have the word "short" in the name, then they are "long" ETF, while ones with word "short" are short ETF. So, in a bull market, short ETF suffer badly, while in a bear market, they give great return. Since we are investing in the stock market for ever, "short" ETF have no place in our portfolio (they will eventually go down to zero).

1. S&P 500: ProShares Ultra S&P500 (SSO) is 2X leveraged, while ProShares UltraPro S&P500 (UPRO) is 3X leveraged version of SPY. Their expense ratio is 0.93% for both, and dividend is close to 0.01%.

2. Nasdaq 100: ProShares Ultra QQQ (QLD) is 2X leveraged, while ProShares UltraPro QQQ (TQQQ) is 3X leveraged version of QQQ. Their expense ratio is 0.95% for both, and dividend is close to 0.01%.

If you really have to invest in leveraged ETF, TQQQ is the best option of all, as it's most highly traded leveraged ETF (with $18B in AUM), and allows you to maximize your return by going all in, basically saying insanity is not here to stay but to keep growing. But buy these ONLY when Nasdaq itself is down by 30% or more from the peak.

 


 

Which index funds to stick to?

So, now that we have looked at index ETF and MF, which index should we try to mimic? Let's see below.

SPY ETF:

Below is the dividend history of SPY ETF, which is longest one to track S&P500: https://www.dividend.com/etfs/spy-spdr-s-p-500-etf-trust/

In 1994, dividend for SPY was $1.17/unit, while in 2021, dividend was $5.65/unit. So, dividend went up almost 5X in 28 years, implying an average annual growth of 5%-6% in dividend. In 2000, dividend was $1.45/unit. while in 2010, dividend was $2.2/unit. So, dividend used to grow by 4% a year before 2010, but since 2010, it has really sped up, and increasing at a rate of 8% a year. This means, more and more companies are maxing out on their dividend, and room for further dividend growth is getting smaller. This is unsustainable dividend growth. Dividend can grow only by same rate as GDP, as revenues and profit increase by same rate as GDP, so it's logical that dividends would also go up by same rate. Also, other thing to note is that dividend used to yield 2.5%/unit (price=$45/unit in 1994) in 1990s, but now it yields 1.25%/unit (price=$450/unit as of 2021). This implies that not only have dividends been growing faster, but stock prices have been growing even faster (dividends went up 5X while stock prices went up 10X).

QQQ ETF:

Below is the dividend history of QQQ ETF, which is longest one to track Nasdaq100: https://www.dividend.com/etfs/qqq-invesco-qqq-trust/

Unfortunately it shows dividend data starting from 2008, even though this was launched in 1999. In 2008, dividend was $0.14/unit, while in 2021, dividend was $1.7/unit. So, it went 12X in 14 years, implying an average annual growth of 20% in dividend, which is insane. The dividend yield used to be paltry 0.3%/unit (price=$40/unit in 2008), but even today it yields a lowly 0.4%/unit (price=$400/unit in 2021). So, QQQ yield has gone up, in contrast to SPY yield which has gone down. Meanwhile, QQQ has also gone up 10X in last 15 years, while it took SPY 30 years to get to 10X. So, QQQ has been growing much faster than SPY, and yields between SPY and QQQ are narrowing at the same time.

Conclusion: So, it would make sense to invest into both S&P500 and Nasdaq100. The best way to start would be to mimic S&P500 index. Once S&P500 has crashed 20% from the peak, we should start reducing S&P500 index and start increasing Nasdaq100 index. At 40% crash of S&P500 index, we should be investing 100% into Nasdaq100 index. Between 20% to 40% crash of S&P500 index, we should linearly go from 100% to 0% for S&P500, and from 0% to 100% for Nasdaq100 index. This would give a good mix of dividend yield and stock price appreciation.

So, the 3 ETF to stick to are:

  1. QQQM: High price appreciation, but low dividend of 0.5%. However, QQQM gave you the highest return over the last 30 years (1%/yr more return than VOO).
  2. VOO: Tracks S&P500, with decent dividend of 1.5%. VOO has given decent return of 10% over the last 40 years.
  3. SCHD: Low price appreciation, but high dividend of 4%. SCHD has given the least return over the last 15 years (1%/yr less return than VOO). SCHD is more for people nearing retirement, who are looking for a huge cash flow so that they don't have to sell any stocks. With $2.5M invested in SCHD in retirement, you can get $100K in dividends which might suffice for decent life. With QQQM and VOO portfolio, you'll need upwards of $10M to retire.

 

Individual stocks:

After a long rambling on why not to buy individual stock, there may still be the testosterone gushing in some of us who to try something manly to be proud of. In that case, it's OK to invest < 10% of your total holdings in individual stocks. Also, never invest > 1% of your portfolio in a single stock, no matter how much of a slam dunk it appears. We are NOT warren buffet. Always assume that you may completely lose money on this 10% (a single company going bankrupt will never take our portfolio down by > 1%). But we'll make up for the loss by remaining invested in whole of the market in the remaining 90%, which will always overcome the worst case 10% loss, and take us back to positive return.

I maintain a WatchList of stocks that look decent, strong companies with low chance of going bankrupt. Look under "WatchList" section.

Selected Stocks to buy: https://www.simplysafedividends.com/intelligent-income/posts/1-living-off-dividends-in-retirement

According to the Wall Street Journal, over the past 50 years the S&P 500’s dividends grew at an average 5.7% per year, outpacing the average 4.1% inflation rate.

https://www.simplysafedividends.com/intelligent-income/posts/6-dividend-aristocrats

53 companies listed here. Most of them have dividends > 2%.

 ---------------

Wishing you a Happy participation in the biggest Govt run Ponzi scheme !!